2019-12-09T08:50:35.859 INFO:root:teuthology version: 1.0.0-8b630f9 2019-12-09T08:50:35.860 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.ylyqhP.tmp -- --verbose --lock --description rgw/verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} --name ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi --block --owner scheduled_ofriedma@teuthology --archive /home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584190 2019-12-09T08:50:35.917 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T08:50:35.989 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584190 branch: wip-ofriedma-dynamic-disable description: rgw/verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} email: ofriedma@redhat.com first_in_suite: false job_id: '4584190' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos overrides: admin_socket: branch: wip-ofriedma-dynamic-disable ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw lc debug interval: 10 global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 osd fast shutdown: false osd objectstore: filestore osd shutdown pgref assert: true osd sloppy crc: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME sha1: 1351c819e1a9518cde4c8b12d645782009028360 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: flavor: notcmalloc sha1: 1351c819e1a9518cde4c8b12d645782009028360 rgw: compression type: random ec-data-pool: true frontend: civetweb storage classes: LUKEWARM, FROZEN s3tests: slow_backend: true workunit: branch: wip-ofriedma-dynamic-disable sha1: 1351c819e1a9518cde4c8b12d645782009028360 owner: scheduled_ofriedma@teuthology priority: 60 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 sha1: 1351c819e1a9518cde4c8b12d645782009028360 suite: rgw suite_branch: wip-ofriedma-dynamic-disable suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 1351c819e1a9518cde4c8b12d645782009028360 tasks: - install: flavor: notcmalloc - ceph: null - openssl_keys: null - rgw: client.0: valgrind: - --tool=memcheck - --max-threads=1024 - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - rgw/test_rgw_gc_log.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - s3tests-java: client.0: force-branch: master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: force-branch: ceph-master rgw_server: client.0 - swift: client.0: rgw_server: client.0 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.22738 2019-12-09T08:50:35.990 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa; will attempt to use it 2019-12-09T08:50:35.990 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks 2019-12-09T08:50:35.991 INFO:teuthology.run_tasks:Running task internal.check_packages... 2019-12-09T08:50:35.994 INFO:teuthology.task.internal:Checking packages... 2019-12-09T08:50:36.009 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'notcmalloc' and ceph hash '1351c819e1a9518cde4c8b12d645782009028360' 2019-12-09T08:50:36.009 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-09T08:50:36.009 INFO:teuthology.packaging:ref: None 2019-12-09T08:50:36.009 INFO:teuthology.packaging:tag: None 2019-12-09T08:50:36.009 INFO:teuthology.packaging:branch: wip-ofriedma-dynamic-disable 2019-12-09T08:50:36.010 INFO:teuthology.packaging:sha1: 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T08:50:36.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&ref=wip-ofriedma-dynamic-disable 2019-12-09T08:50:36.176 INFO:teuthology.task.internal:Found packages for ceph version 15.0.0-7900.g1351c81 2019-12-09T08:50:36.176 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2019-12-09T08:50:36.255 INFO:teuthology.task.internal:no buildpackages task found 2019-12-09T08:50:36.255 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2019-12-09T08:50:36.282 INFO:teuthology.task.internal.lock_machines:Locking machines... 2019-12-09T08:50:36.282 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T08:50:36.337 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T08:50:46.355 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T08:50:56.372 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T08:51:06.392 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T08:51:16.409 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:51:26.425 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:51:36.438 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:51:46.457 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:51:56.474 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:52:06.496 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:52:16.581 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:52:26.601 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:52:36.610 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:52:46.635 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:52:56.653 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:53:06.672 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:53:16.690 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:53:26.708 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:53:36.726 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:53:46.740 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:53:56.756 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:54:06.776 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:54:16.793 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:54:26.815 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:54:36.832 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T08:54:46.852 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T08:54:56.873 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T08:55:06.890 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T08:55:16.906 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T08:55:26.921 DEBUG:teuthology.lock.ops:lock_many request: {'count': 2, 'locked_by': 'scheduled_ofriedma@teuthology', 'description': '/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584190', 'machine_type': 'smithi'} 2019-12-09T08:55:26.938 DEBUG:teuthology.lock.ops:locked ubuntu@smithi022.front.sepia.ceph.com, ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T08:55:27.017 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-09T08:55:27.216 INFO:teuthology.provision.fog.smithi035:Scheduling deploy of centos 7.6 2019-12-09T08:55:27.219 INFO:teuthology.provision.fog.smithi022:Scheduling deploy of centos 7.6 2019-12-09T08:55:27.589 INFO:teuthology.orchestra.console:Power off smithi022 2019-12-09T08:55:27.591 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-09T08:55:27.635 INFO:teuthology.orchestra.console:Power off smithi035 2019-12-09T08:55:27.635 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi035.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-09T08:55:27.649 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:27.675 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi035.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:31.712 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:31.722 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi035.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:35.824 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:35.843 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi035.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:36.073 INFO:teuthology.orchestra.console:Power off for smithi035 completed 2019-12-09T08:55:36.174 INFO:teuthology.orchestra.console:Power on smithi035 2019-12-09T08:55:36.175 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi035.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-09T08:55:36.209 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi035.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:39.948 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:40.160 INFO:teuthology.orchestra.console:Power off for smithi022 completed 2019-12-09T08:55:40.246 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi035.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:40.262 INFO:teuthology.orchestra.console:Power on smithi022 2019-12-09T08:55:40.263 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-09T08:55:40.290 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:44.342 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:44.362 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi035.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T08:55:44.553 INFO:teuthology.orchestra.console:Power on for smithi022 completed 2019-12-09T08:55:44.575 INFO:teuthology.orchestra.console:Power on for smithi035 completed 2019-12-09T08:55:44.655 INFO:teuthology.provision.fog.smithi022:Waiting for deploy to finish 2019-12-09T08:55:44.676 INFO:teuthology.provision.fog.smithi035:Waiting for deploy to finish 2019-12-09T08:58:00.676 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:58:09.805 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:58:15.753 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:58:18.858 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:58:24.811 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:58:28.016 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:58:34.220 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:58:37.100 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:58:43.274 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:58:46.154 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:58:52.337 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:58:55.214 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:59:01.420 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:59:04.266 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:59:10.477 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:59:19.537 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:59:57.101 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T08:59:57.319 INFO:teuthology.orchestra.run.smithi022:> hostname 2019-12-09T08:59:57.803 INFO:teuthology.orchestra.run.smithi022.stdout:smithi068 2019-12-09T08:59:57.804 INFO:teuthology.orchestra.run.smithi022:> grep smithi068 /etc/hosts 2019-12-09T08:59:57.846 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.68 smithi068.front.sepia.ceph.com smithi068 2019-12-09T08:59:57.846 INFO:teuthology.orchestra.run.smithi022:> sudo hostname smithi022 2019-12-09T08:59:57.965 INFO:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi068/smithi022/g' /etc/hosts 2019-12-09T08:59:58.025 INFO:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/172.21.15.68/172.21.15.22/g' /etc/hosts 2019-12-09T08:59:58.080 INFO:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi068/smithi022/g' /etc/hostname 2019-12-09T08:59:58.133 INFO:teuthology.orchestra.run.smithi022:> sudo hostname smithi022 2019-12-09T08:59:58.185 INFO:teuthology.provision.fog.smithi022:Deploy complete! 2019-12-09T08:59:58.187 DEBUG:teuthology.parallel:result is None 2019-12-09T09:00:10.278 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:00:10.481 INFO:teuthology.orchestra.run.smithi035:> hostname 2019-12-09T09:00:11.042 INFO:teuthology.orchestra.run.smithi035.stdout:smithi035.front.sepia.ceph.com 2019-12-09T09:00:11.042 INFO:teuthology.orchestra.run.smithi035:> grep smithi035.front.sepia.ceph.com /etc/hosts 2019-12-09T09:00:11.129 INFO:teuthology.orchestra.run.smithi035.stdout:172.21.15.35 smithi035.front.sepia.ceph.com smithi035 2019-12-09T09:00:11.129 INFO:teuthology.orchestra.run.smithi035:> sudo hostname smithi035 2019-12-09T09:00:11.289 INFO:teuthology.orchestra.run.smithi035:> sudo sed -i -e 's/smithi035.front.sepia.ceph.com/smithi035/g' /etc/hosts 2019-12-09T09:00:11.399 INFO:teuthology.orchestra.run.smithi035:> sudo sed -i -e 's/172.21.15.35/172.21.15.35/g' /etc/hosts 2019-12-09T09:00:11.489 INFO:teuthology.orchestra.run.smithi035:> sudo sed -i -e 's/smithi035.front.sepia.ceph.com/smithi035/g' /etc/hostname 2019-12-09T09:00:11.570 INFO:teuthology.orchestra.run.smithi035:> sudo hostname smithi035 2019-12-09T09:00:11.636 INFO:teuthology.provision.fog.smithi035:Deploy complete! 2019-12-09T09:00:11.637 DEBUG:teuthology.parallel:result is None 2019-12-09T09:00:11.833 INFO:teuthology.lock.ops:Checking smithi035.front.sepia.ceph.com 2019-12-09T09:00:11.834 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-09T09:00:11.851 INFO:teuthology.lock.ops:Checking smithi022.front.sepia.ceph.com 2019-12-09T09:00:11.851 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-09T09:00:11.875 INFO:teuthology.lock.ops:Updating [smithi035.front.sepia.ceph.com]: set os type and version on server 2019-12-09T09:00:11.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:00:11.876 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:00:11.976 INFO:teuthology.orchestra.run.smithi035:> true 2019-12-09T09:00:12.357 INFO:teuthology.orchestra.run.smithi035:> uname -m 2019-12-09T09:00:12.422 INFO:teuthology.orchestra.run.smithi035.stdout:x86_64 2019-12-09T09:00:12.422 INFO:teuthology.orchestra.run.smithi035:> cat /etc/os-release 2019-12-09T09:00:12.497 INFO:teuthology.orchestra.run.smithi035.stdout:NAME="CentOS Linux" 2019-12-09T09:00:12.498 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION="7 (Core)" 2019-12-09T09:00:12.498 INFO:teuthology.orchestra.run.smithi035.stdout:ID="centos" 2019-12-09T09:00:12.498 INFO:teuthology.orchestra.run.smithi035.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:00:12.498 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION_ID="7" 2019-12-09T09:00:12.498 INFO:teuthology.orchestra.run.smithi035.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:00:12.498 INFO:teuthology.orchestra.run.smithi035.stdout:ANSI_COLOR="0;31" 2019-12-09T09:00:12.498 INFO:teuthology.orchestra.run.smithi035.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:00:12.499 INFO:teuthology.orchestra.run.smithi035.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:00:12.499 INFO:teuthology.orchestra.run.smithi035.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:00:12.499 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:00:12.499 INFO:teuthology.orchestra.run.smithi035.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:00:12.499 INFO:teuthology.orchestra.run.smithi035.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:00:12.499 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:00:12.499 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:00:12.499 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:00:12.502 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server 2019-12-09T09:00:12.521 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: set os type and version on server 2019-12-09T09:00:12.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:00:12.529 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:00:12.628 INFO:teuthology.orchestra.run.smithi022:> true 2019-12-09T09:00:13.033 INFO:teuthology.orchestra.run.smithi022:> uname -m 2019-12-09T09:00:13.121 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2019-12-09T09:00:13.122 INFO:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2019-12-09T09:00:13.192 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Linux" 2019-12-09T09:00:13.192 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="7 (Core)" 2019-12-09T09:00:13.192 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2019-12-09T09:00:13.193 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:00:13.193 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="7" 2019-12-09T09:00:13.193 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:00:13.193 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2019-12-09T09:00:13.193 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:00:13.193 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:00:13.194 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:00:13.194 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:00:13.194 INFO:teuthology.orchestra.run.smithi022.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:00:13.194 INFO:teuthology.orchestra.run.smithi022.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:00:13.194 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:00:13.194 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:00:13.195 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:00:13.195 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2019-12-09T09:00:13.210 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2019-12-09T09:00:13.242 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi022.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4hazoKCZRee8MFRtc7S7yQdf1IxkTxCzrh/vNl69HpJJxXzCBQ3yBjc+QdjJYo2us10/vk+/JdIdHhuwEqQ/6lEJZMLsj6veaoX5Ugr8F9AHPD2K9kv3g9T3rURL/oM2oGGFwNe+lc8BdsVyEqVt3VxmkLniK5kci5rwkFCf8xek8qt3AmMdEMZBMsMPclJ9NW2Xw4KCvYMXoTbPpVs5S2bFHTCua9RBjRPqRr+3vBnLx3PKyIIMaT7oNerbszXORGFQWH3RNbLfinhFQIonxR+7BqezG+kn2d8pv38SJPT0Pf84WSN1NN1drfybCIVKXkDpP5oT9ZCWtVU0E6ZDX smithi035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCotl1Z+PSkiOfwNCfW9Np2KP4HeR18WJ23R7YL66Fu6Moyya5PRbhv4wvQtBXiof0aAQnF55aHkOBO4MaeqynaSwnnjhR1eoT6LRT1Jakg1egN6IMLZnBaBJyfPXdGy/DPfnhghNunyUGAhDwREosZXd0Ur6BDcpg2NRNzhR0Rub4oZInZ7yMqbTJsEQTtguXEfkR4aG02dfcHQecQ5BK/10zn7uBNrRZZWLGYTwCosT5M9hbs4N8y9oEmBDYTtG6ZPCL5OgVdrNDuQykuA2IIqT0aq35hOhm8WJSaGnoMwirsjVSbmGNoB4ZCEzY45RGB+mEQK/lES39B60MChtnv 2019-12-09T09:00:13.243 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T09:00:13.294 INFO:teuthology.run_tasks:Running task internal.save_config... 2019-12-09T09:00:13.300 INFO:teuthology.task.internal:Saving configuration 2019-12-09T09:00:13.319 INFO:teuthology.run_tasks:Running task internal.check_lock... 2019-12-09T09:00:13.332 INFO:teuthology.task.internal.check_lock:Checking locks... 2019-12-09T09:00:13.347 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-09 08:55:26.929439', u'locked_by': u'scheduled_ofriedma@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi035.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAHpJOTy4L+I6GcrYNSJVywf5v74+GfWhxWF22pYpupqBBNLEaLwdVAFR730cJFxp34yegwHN/kljv8vNaualxk=', u'description': u'/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584190'} 2019-12-09T09:00:13.363 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-09 08:55:26.929476', u'locked_by': u'scheduled_ofriedma@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi022.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFhBQLramVnzGowR2SSAAQSFmmzK4lhOubux972EViY68RMRtZAsUuPkMY0GFYZ7GAdD9jChKZ5vyfnkF0BWJc=', u'description': u'/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584190'} 2019-12-09T09:00:13.363 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2019-12-09T09:00:13.371 INFO:teuthology.task.internal:roles: ubuntu@smithi035.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-09T09:00:13.371 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-12-09T09:00:13.371 INFO:teuthology.run_tasks:Running task console_log... 2019-12-09T09:00:13.457 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-09T09:00:13.458 INFO:teuthology.run_tasks:Running task internal.connect... 2019-12-09T09:00:13.504 INFO:teuthology.task.internal:Opening connections... 2019-12-09T09:00:13.505 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:00:13.506 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:00:13.605 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2019-12-09T09:00:13.606 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:00:13.708 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2019-12-09T09:00:13.718 INFO:teuthology.orchestra.run.smithi035:> uname -m 2019-12-09T09:00:13.922 INFO:teuthology.orchestra.run.smithi035.stdout:x86_64 2019-12-09T09:00:13.922 INFO:teuthology.orchestra.run.smithi035:> cat /etc/os-release 2019-12-09T09:00:13.993 INFO:teuthology.orchestra.run.smithi035.stdout:NAME="CentOS Linux" 2019-12-09T09:00:13.993 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION="7 (Core)" 2019-12-09T09:00:13.993 INFO:teuthology.orchestra.run.smithi035.stdout:ID="centos" 2019-12-09T09:00:13.994 INFO:teuthology.orchestra.run.smithi035.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:00:13.994 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION_ID="7" 2019-12-09T09:00:13.994 INFO:teuthology.orchestra.run.smithi035.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:00:13.994 INFO:teuthology.orchestra.run.smithi035.stdout:ANSI_COLOR="0;31" 2019-12-09T09:00:13.994 INFO:teuthology.orchestra.run.smithi035.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:00:13.994 INFO:teuthology.orchestra.run.smithi035.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:00:13.994 INFO:teuthology.orchestra.run.smithi035.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:00:13.995 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:00:13.995 INFO:teuthology.orchestra.run.smithi035.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:00:13.995 INFO:teuthology.orchestra.run.smithi035.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:00:13.995 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:00:13.995 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:00:13.995 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:00:13.996 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server 2019-12-09T09:00:14.014 INFO:teuthology.orchestra.run.smithi022:> uname -m 2019-12-09T09:00:14.142 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2019-12-09T09:00:14.143 INFO:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2019-12-09T09:00:14.219 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Linux" 2019-12-09T09:00:14.219 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="7 (Core)" 2019-12-09T09:00:14.219 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2019-12-09T09:00:14.219 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:00:14.219 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="7" 2019-12-09T09:00:14.220 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:00:14.220 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2019-12-09T09:00:14.220 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:00:14.220 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:00:14.220 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:00:14.220 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:00:14.220 INFO:teuthology.orchestra.run.smithi022.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:00:14.221 INFO:teuthology.orchestra.run.smithi022.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:00:14.221 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:00:14.221 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:00:14.221 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:00:14.221 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2019-12-09T09:00:14.239 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2019-12-09T09:00:14.278 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2019-12-09T09:00:14.289 INFO:teuthology.task.internal:Checking for old test directory... 2019-12-09T09:00:14.289 INFO:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2019-12-09T09:00:14.294 INFO:teuthology.orchestra.run.smithi035:> test '!' -e /home/ubuntu/cephtest 2019-12-09T09:00:14.382 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2019-12-09T09:00:14.393 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2019-12-09T09:00:14.394 INFO:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2019-12-09T09:00:14.397 INFO:teuthology.orchestra.run.smithi035:> test -z $(ls -A /var/lib/ceph) 2019-12-09T09:00:14.486 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2019-12-09T09:00:14.531 INFO:teuthology.run_tasks:Running task kernel... 2019-12-09T09:00:14.550 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'osd.3': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'osd.5': {'sha1': 'distro', 'kdb': True}, 'osd.4': {'sha1': 'distro', 'kdb': True}, 'osd.7': {'sha1': 'distro', 'kdb': True}, 'osd.6': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}} 2019-12-09T09:00:14.551 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}}, timeout 300 2019-12-09T09:00:14.551 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:00:14.551 INFO:teuthology.orchestra.run.smithi035:> sudo package-cleanup -y --oldkernels 2019-12-09T09:00:15.420 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:00:16.656 INFO:teuthology.orchestra.run.smithi035.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:00:16.656 INFO:teuthology.orchestra.run.smithi035.stdout:Trying other mirror. 2019-12-09T09:00:16.657 INFO:teuthology.orchestra.run.smithi035.stdout:To address this issue please refer to the below wiki article 2019-12-09T09:00:16.657 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:00:16.657 INFO:teuthology.orchestra.run.smithi035.stdout:https://wiki.centos.org/yum-errors 2019-12-09T09:00:16.657 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:00:16.657 INFO:teuthology.orchestra.run.smithi035.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-12-09T09:00:16.658 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:00:16.803 INFO:teuthology.orchestra.run.smithi035.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:00:16.803 INFO:teuthology.orchestra.run.smithi035.stdout:Trying other mirror. 2019-12-09T09:00:16.895 INFO:teuthology.orchestra.run.smithi035.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:00:16.895 INFO:teuthology.orchestra.run.smithi035.stdout:Trying other mirror. 2019-12-09T09:00:17.038 INFO:teuthology.orchestra.run.smithi035.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:00:17.038 INFO:teuthology.orchestra.run.smithi035.stdout:Trying other mirror. 2019-12-09T09:00:17.191 INFO:teuthology.orchestra.run.smithi035.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:00:17.191 INFO:teuthology.orchestra.run.smithi035.stdout:Trying other mirror. 2019-12-09T09:00:17.811 INFO:teuthology.orchestra.run.smithi035.stdout:No old kernels to remove 2019-12-09T09:00:17.829 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi022.front.sepia.ceph.com 2019-12-09T09:00:17.829 INFO:teuthology.orchestra.run.smithi022:> sudo package-cleanup -y --oldkernels 2019-12-09T09:00:18.714 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:00:19.671 INFO:teuthology.orchestra.run.smithi022.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:00:19.671 INFO:teuthology.orchestra.run.smithi022.stdout:Trying other mirror. 2019-12-09T09:00:19.671 INFO:teuthology.orchestra.run.smithi022.stdout:To address this issue please refer to the below wiki article 2019-12-09T09:00:19.671 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:00:19.671 INFO:teuthology.orchestra.run.smithi022.stdout:https://wiki.centos.org/yum-errors 2019-12-09T09:00:19.671 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:00:19.672 INFO:teuthology.orchestra.run.smithi022.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-12-09T09:00:19.672 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:00:19.817 INFO:teuthology.orchestra.run.smithi022.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:00:19.817 INFO:teuthology.orchestra.run.smithi022.stdout:Trying other mirror. 2019-12-09T09:00:19.909 INFO:teuthology.orchestra.run.smithi022.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:00:19.909 INFO:teuthology.orchestra.run.smithi022.stdout:Trying other mirror. 2019-12-09T09:00:20.052 INFO:teuthology.orchestra.run.smithi022.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:00:20.052 INFO:teuthology.orchestra.run.smithi022.stdout:Trying other mirror. 2019-12-09T09:00:20.205 INFO:teuthology.orchestra.run.smithi022.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:00:20.205 INFO:teuthology.orchestra.run.smithi022.stdout:Trying other mirror. 2019-12-09T09:00:20.779 INFO:teuthology.orchestra.run.smithi022.stdout:No old kernels to remove 2019-12-09T09:00:20.802 INFO:teuthology.orchestra.run.smithi035:> uname -r 2019-12-09T09:00:20.909 INFO:teuthology.orchestra.run.smithi035.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:00:20.909 INFO:teuthology.task.kernel:Running kernel on smithi035: 3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:00:20.910 INFO:teuthology.orchestra.run.smithi035:> sudo yum install -y kernel 2019-12-09T09:00:21.207 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:00:21.252 INFO:teuthology.orchestra.run.smithi035.stdout:Determining fastest mirrors 2019-12-09T09:00:21.406 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:00:21.406 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: ftp.linux.ncsu.edu 2019-12-09T09:00:21.407 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:00:21.409 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:00:22.914 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:00:22.914 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:00:22.915 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-09T09:00:24.346 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:00:25.151 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:00:25.151 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:00:25.165 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:00:25.165 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:00:25.165 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2019-12-09T09:00:25.166 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:00:25.166 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:00:25.166 INFO:teuthology.orchestra.run.smithi035.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-09T09:00:25.166 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:00:25.166 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:00:25.166 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:00:25.167 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2019-12-09T09:00:25.167 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:00:25.167 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 50 M 2019-12-09T09:00:25.167 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 64 M 2019-12-09T09:00:25.167 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:00:25.168 INFO:teuthology.orchestra.run.smithi035.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:00:26.369 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:00:26.566 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:00:27.483 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:00:27.484 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:01:04.178 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:01:04.682 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:01:04.682 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:01:04.682 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:01:04.682 INFO:teuthology.orchestra.run.smithi035.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-09T09:01:04.682 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:01:04.683 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:01:04.738 INFO:teuthology.orchestra.run.smithi035:> rpm -q kernel --last | head -n 1 2019-12-09T09:01:05.148 INFO:teuthology.orchestra.run.smithi035.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:00:30 AM UTC 2019-12-09T09:01:05.150 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:01:05.151 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:01:05.151 INFO:teuthology.orchestra.run.smithi022:> uname -r 2019-12-09T09:01:05.273 INFO:teuthology.orchestra.run.smithi022.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:01:05.274 INFO:teuthology.task.kernel:Running kernel on smithi022: 3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:01:05.274 INFO:teuthology.orchestra.run.smithi022:> sudo yum install -y kernel 2019-12-09T09:01:05.542 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:01:05.584 INFO:teuthology.orchestra.run.smithi022.stdout:Determining fastest mirrors 2019-12-09T09:01:06.682 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:01:06.683 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: ftp.linux.ncsu.edu 2019-12-09T09:01:06.684 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:01:06.685 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: centos.mirror.constant.com 2019-12-09T09:01:08.453 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:01:08.453 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:01:08.454 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-09T09:01:09.800 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:01:10.644 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:01:10.644 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:01:10.654 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:01:10.654 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:01:10.654 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2019-12-09T09:01:10.655 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:01:10.655 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:01:10.655 INFO:teuthology.orchestra.run.smithi022.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-09T09:01:10.655 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:01:10.655 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:01:10.655 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:01:10.656 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2019-12-09T09:01:10.656 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:01:10.656 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 50 M 2019-12-09T09:01:10.656 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 64 M 2019-12-09T09:01:10.656 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:01:10.661 INFO:teuthology.orchestra.run.smithi022.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:01:11.833 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:01:12.033 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:01:12.842 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:01:12.842 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:01:49.772 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:01:50.338 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:01:50.339 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:01:50.339 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:01:50.339 INFO:teuthology.orchestra.run.smithi022.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-09T09:01:50.339 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:01:50.339 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:01:50.389 INFO:teuthology.orchestra.run.smithi022:> rpm -q kernel --last | head -n 1 2019-12-09T09:01:50.779 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:01:15 AM UTC 2019-12-09T09:01:50.782 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:01:50.783 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:01:50.783 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2019-12-09T09:01:50.783 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-09T09:01:50.783 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-09T09:01:50.783 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2019-12-09T09:01:50.784 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi035.front.sepia.ceph.com, path=None, version=distro) 2019-12-09T09:01:50.784 INFO:teuthology.orchestra.run.smithi035:> rpm -q kernel --last | head -n 1 2019-12-09T09:01:51.161 INFO:teuthology.orchestra.run.smithi035.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:00:30 AM UTC 2019-12-09T09:01:51.164 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:01:51.164 INFO:teuthology.orchestra.run.smithi035:> sudo rpm -qi grub2 2019-12-09T09:01:51.333 INFO:teuthology.orchestra.run.smithi035.stdout:Name : grub2 2019-12-09T09:01:51.333 INFO:teuthology.orchestra.run.smithi035.stdout:Epoch : 1 2019-12-09T09:01:51.333 INFO:teuthology.orchestra.run.smithi035.stdout:Version : 2.02 2019-12-09T09:01:51.334 INFO:teuthology.orchestra.run.smithi035.stdout:Release : 0.76.el7.centos 2019-12-09T09:01:51.334 INFO:teuthology.orchestra.run.smithi035.stdout:Architecture: x86_64 2019-12-09T09:01:51.334 INFO:teuthology.orchestra.run.smithi035.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-12-09T09:01:51.334 INFO:teuthology.orchestra.run.smithi035.stdout:Group : System Environment/Base 2019-12-09T09:01:51.334 INFO:teuthology.orchestra.run.smithi035.stdout:Size : 0 2019-12-09T09:01:51.334 INFO:teuthology.orchestra.run.smithi035.stdout:License : GPLv3+ 2019-12-09T09:01:51.334 INFO:teuthology.orchestra.run.smithi035.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-09T09:01:51.335 INFO:teuthology.orchestra.run.smithi035.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-09T09:01:51.335 INFO:teuthology.orchestra.run.smithi035.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-09T09:01:51.335 INFO:teuthology.orchestra.run.smithi035.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-09T09:01:51.335 INFO:teuthology.orchestra.run.smithi035.stdout:Relocations : (not relocatable) 2019-12-09T09:01:51.335 INFO:teuthology.orchestra.run.smithi035.stdout:Packager : CentOS BuildSystem 2019-12-09T09:01:51.335 INFO:teuthology.orchestra.run.smithi035.stdout:Vendor : CentOS 2019-12-09T09:01:51.336 INFO:teuthology.orchestra.run.smithi035.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-09T09:01:51.336 INFO:teuthology.orchestra.run.smithi035.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-12-09T09:01:51.336 INFO:teuthology.orchestra.run.smithi035.stdout:Description : 2019-12-09T09:01:51.336 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:01:51.340 INFO:teuthology.orchestra.run.smithi035.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-09T09:01:51.341 INFO:teuthology.orchestra.run.smithi035.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-09T09:01:51.341 INFO:teuthology.orchestra.run.smithi035.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-09T09:01:51.341 INFO:teuthology.orchestra.run.smithi035.stdout:hardware devices. 2019-12-09T09:01:51.352 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-09T09:01:51.352 INFO:teuthology.task.kernel:Updating grub on smithi035 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:01:51.353 INFO:teuthology.orchestra.run.smithi035:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-09T09:01:51.871 INFO:teuthology.orchestra.run.smithi035.stderr:Generating grub configuration file ... 2019-12-09T09:01:52.011 INFO:teuthology.orchestra.run.smithi035.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:01:52.027 INFO:teuthology.orchestra.run.smithi035.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-09T09:01:52.329 INFO:teuthology.orchestra.run.smithi035.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:01:52.349 INFO:teuthology.orchestra.run.smithi035.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-09T09:01:52.386 INFO:teuthology.orchestra.run.smithi035.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-09T09:01:52.408 INFO:teuthology.orchestra.run.smithi035.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-09T09:01:52.441 INFO:teuthology.orchestra.run.smithi035.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-09T09:01:52.453 INFO:teuthology.orchestra.run.smithi035.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-09T09:01:54.006 INFO:teuthology.orchestra.run.smithi035.stderr:done 2019-12-09T09:01:54.008 INFO:teuthology.orchestra.run.smithi035:> mktemp 2019-12-09T09:01:54.083 INFO:teuthology.orchestra.run.smithi035.stdout:/tmp/tmp.kkdbuu8Ysc 2019-12-09T09:01:54.084 INFO:teuthology.orchestra.run.smithi035:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kkdbuu8Ysc 2019-12-09T09:01:54.200 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 0666 /tmp/tmp.kkdbuu8Ysc 2019-12-09T09:01:54.406 DEBUG:teuthology.orchestra.remote:smithi035:/tmp/tmp.kkdbuu8Ysc is 6KB 2019-12-09T09:01:54.499 INFO:teuthology.orchestra.run.smithi035:> rm -fr /tmp/tmp.kkdbuu8Ysc 2019-12-09T09:01:54.562 INFO:teuthology.orchestra.run.smithi035:> sudo grub2-set-default 0 2019-12-09T09:01:54.735 INFO:teuthology.orchestra.run.smithi035:> sudo shutdown -r now 2019-12-09T09:01:54.739 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2019-12-09T09:01:54.739 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2019-12-09T09:01:54.739 INFO:teuthology.orchestra.run.smithi022:> rpm -q kernel --last | head -n 1 2019-12-09T09:01:55.094 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:01:15 AM UTC 2019-12-09T09:01:55.096 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:01:55.097 INFO:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2 2019-12-09T09:01:55.278 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2 2019-12-09T09:01:55.279 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2019-12-09T09:01:55.279 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.02 2019-12-09T09:01:55.279 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 0.76.el7.centos 2019-12-09T09:01:55.280 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2019-12-09T09:01:55.280 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-12-09T09:01:55.280 INFO:teuthology.orchestra.run.smithi022.stdout:Group : System Environment/Base 2019-12-09T09:01:55.280 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 0 2019-12-09T09:01:55.280 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2019-12-09T09:01:55.281 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-09T09:01:55.281 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-09T09:01:55.281 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-09T09:01:55.281 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-09T09:01:55.282 INFO:teuthology.orchestra.run.smithi022.stdout:Relocations : (not relocatable) 2019-12-09T09:01:55.282 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : CentOS BuildSystem 2019-12-09T09:01:55.282 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : CentOS 2019-12-09T09:01:55.282 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-09T09:01:55.283 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-12-09T09:01:55.283 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2019-12-09T09:01:55.283 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:01:55.283 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-09T09:01:55.283 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-09T09:01:55.284 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-09T09:01:55.284 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2019-12-09T09:01:55.285 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-09T09:01:55.285 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:01:55.285 INFO:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-09T09:01:55.860 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2019-12-09T09:01:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:01:56.048 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-09T09:01:56.321 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:01:56.343 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-09T09:01:56.378 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-09T09:01:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-09T09:01:56.424 INFO:teuthology.orchestra.run.smithi022.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-09T09:01:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-09T09:01:57.388 INFO:teuthology.orchestra.run.smithi022.stderr:done 2019-12-09T09:01:57.391 INFO:teuthology.orchestra.run.smithi022:> mktemp 2019-12-09T09:01:57.467 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.d59XfVgR9g 2019-12-09T09:01:57.467 INFO:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.d59XfVgR9g 2019-12-09T09:01:57.556 INFO:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.d59XfVgR9g 2019-12-09T09:01:57.711 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.d59XfVgR9g is 6KB 2019-12-09T09:01:57.795 INFO:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.d59XfVgR9g 2019-12-09T09:01:57.880 INFO:teuthology.orchestra.run.smithi022:> sudo grub2-set-default 0 2019-12-09T09:01:58.070 INFO:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2019-12-09T09:01:58.074 INFO:teuthology.misc:Re-opening connections... 2019-12-09T09:01:58.074 INFO:teuthology.misc:trying to connect to ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:01:58.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:01:58.075 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:01:58.076 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.35 2019-12-09T09:01:58.076 INFO:teuthology.misc:trying to connect to ubuntu@smithi022.front.sepia.ceph.com 2019-12-09T09:01:58.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:01:58.078 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:01:58.172 INFO:teuthology.orchestra.run.smithi022:> true 2019-12-09T09:01:58.565 DEBUG:teuthology.misc:waited 0.490680932999 2019-12-09T09:01:59.566 INFO:teuthology.misc:trying to connect to ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:01:59.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:01:59.567 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:02:59.627 DEBUG:teuthology.orchestra.remote:timed out 2019-12-09T09:02:59.628 DEBUG:teuthology.misc:waited 61.5531659126 2019-12-09T09:03:00.628 INFO:teuthology.misc:trying to connect to ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:03:00.628 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:03:00.629 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:04:00.630 DEBUG:teuthology.orchestra.remote:timed out 2019-12-09T09:04:00.630 DEBUG:teuthology.misc:waited 122.55556488 2019-12-09T09:04:01.630 INFO:teuthology.misc:trying to connect to ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:04:01.631 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:04:01.631 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:04:01.735 INFO:teuthology.orchestra.run.smithi035:> true 2019-12-09T09:04:02.186 DEBUG:teuthology.misc:waited 124.111325979 2019-12-09T09:04:03.187 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2019-12-09T09:04:03.187 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-09T09:04:03.188 INFO:teuthology.orchestra.run.smithi035:> uname -r 2019-12-09T09:04:03.286 INFO:teuthology.orchestra.run.smithi035.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:04:03.286 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:04:03.286 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-09T09:04:03.286 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2019-12-09T09:04:03.286 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-09T09:04:03.287 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:04:03.287 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:04:03.392 INFO:teuthology.orchestra.run.smithi022:> true 2019-12-09T09:04:03.835 INFO:teuthology.orchestra.run.smithi022:> uname -r 2019-12-09T09:04:03.922 INFO:teuthology.orchestra.run.smithi022.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:04:03.923 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:04:03.923 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-09T09:04:04.924 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2019-12-09T09:04:04.925 INFO:teuthology.orchestra.run.smithi035:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-09T09:04:05.087 INFO:teuthology.orchestra.run.smithi035.stdout:ttyS1 2019-12-09T09:04:05.102 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2019-12-09T09:04:05.103 INFO:teuthology.orchestra.run.smithi022:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-09T09:04:05.243 INFO:teuthology.orchestra.run.smithi022.stdout:ttyS1 2019-12-09T09:04:05.255 INFO:teuthology.run_tasks:Running task internal.base... 2019-12-09T09:04:05.261 INFO:teuthology.task.internal:Creating test directory... 2019-12-09T09:04:05.262 INFO:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-09T09:04:05.266 INFO:teuthology.orchestra.run.smithi035:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-09T09:04:05.360 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2019-12-09T09:04:05.368 INFO:teuthology.run_tasks:Running task internal.archive... 2019-12-09T09:04:05.380 INFO:teuthology.task.internal:Creating archive directory... 2019-12-09T09:04:05.380 INFO:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-09T09:04:05.384 INFO:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-09T09:04:05.495 INFO:teuthology.run_tasks:Running task internal.coredump... 2019-12-09T09:04:05.505 INFO:teuthology.task.internal:Enabling coredump saving... 2019-12-09T09:04:05.505 INFO:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:04:05.509 INFO:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:04:05.620 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:04:05.621 INFO:teuthology.orchestra.run.smithi035.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:04:05.624 INFO:teuthology.run_tasks:Running task internal.sudo... 2019-12-09T09:04:05.633 INFO:teuthology.task.internal:Configuring sudo... 2019-12-09T09:04:05.634 INFO:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-09T09:04:05.637 INFO:teuthology.orchestra.run.smithi035:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-09T09:04:05.743 INFO:teuthology.run_tasks:Running task internal.syslog... 2019-12-09T09:04:05.772 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2019-12-09T09:04:05.772 INFO:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-09T09:04:05.776 INFO:teuthology.orchestra.run.smithi035:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-09T09:04:05.864 INFO:teuthology.orchestra.run.smithi035:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:04:05.969 INFO:teuthology.orchestra.run.smithi035:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:04:06.088 INFO:teuthology.orchestra.run.smithi035:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:04:06.179 INFO:teuthology.orchestra.run.smithi035:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:04:06.292 INFO:teuthology.orchestra.run.smithi035:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-09T09:04:06.379 INFO:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:04:06.488 INFO:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:04:06.605 INFO:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:04:06.703 INFO:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:04:06.809 INFO:teuthology.orchestra.run.smithi022:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-09T09:04:06.907 INFO:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2019-12-09T09:04:06.911 INFO:teuthology.orchestra.run.smithi035:> sudo service rsyslog restart 2019-12-09T09:04:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-09T09:04:07.082 INFO:teuthology.orchestra.run.smithi035.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-09T09:04:07.167 INFO:teuthology.run_tasks:Running task internal.timer... 2019-12-09T09:04:07.179 INFO:teuthology.task.internal:Starting timer... 2019-12-09T09:04:07.179 INFO:teuthology.run_tasks:Running task pcp... 2019-12-09T09:04:07.235 INFO:teuthology.run_tasks:Running task selinux... 2019-12-09T09:04:07.249 INFO:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2019-12-09T09:04:07.420 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: [ OK ] 2019-12-09T09:04:07.423 INFO:teuthology.orchestra.run.smithi035:> sudo service auditd rotate 2019-12-09T09:04:07.600 INFO:teuthology.orchestra.run.smithi035.stdout:Rotating logs: [ OK ] 2019-12-09T09:04:07.602 DEBUG:teuthology.task.selinux:Getting current SELinux state 2019-12-09T09:04:07.603 INFO:teuthology.orchestra.run.smithi035:> /usr/sbin/getenforce 2019-12-09T09:04:07.697 INFO:teuthology.orchestra.run.smithi035.stdout:Permissive 2019-12-09T09:04:07.698 INFO:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2019-12-09T09:04:07.800 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2019-12-09T09:04:07.800 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi035.front.sepia.ceph.com': 'permissive'} 2019-12-09T09:04:07.801 INFO:teuthology.orchestra.run.smithi035:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-09T09:04:07.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T09:04:07.925 INFO:teuthology.orchestra.run.smithi022:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-09T09:04:08.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T09:04:08.041 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2019-12-09T09:04:08.041 INFO:teuthology.orchestra.run.smithi035:> sudo /usr/sbin/setenforce permissive 2019-12-09T09:04:08.170 INFO:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2019-12-09T09:04:08.303 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2019-12-09T09:04:08.313 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2019-12-09T09:04:08.314 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2019-12-09T09:04:08.334 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2019-12-09T09:04:08.336 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi035.front.sepia.ceph.com,smithi022.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2019-12-09T09:04:08.754 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2019-12-09T09:04:10.007 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2019-12-09T09:04:12.759 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2019-12-09T09:04:12.784 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:12 +0000 (0:00:00.067) 0:00:00.067 ******* 2019-12-09T09:04:12.797 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:12 +0000 (0:00:00.013) 0:00:00.080 ******* 2019-12-09T09:04:15.408 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2019-12-09T09:04:15.409 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] 2019-12-09T09:04:15.494 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:15 +0000 (0:00:02.697) 0:00:02.777 ******* 2019-12-09T09:04:16.014 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2019-12-09T09:04:16.016 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:04:16.083 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:16 +0000 (0:00:00.588) 0:00:03.365 ******* 2019-12-09T09:04:16.252 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi035.front.sepia.ceph.com] 2019-12-09T09:04:16.337 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:16 +0000 (0:00:00.254) 0:00:03.620 ******* 2019-12-09T09:04:16.688 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2019-12-09T09:04:16.689 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:04:16.725 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:16 +0000 (0:00:00.387) 0:00:04.007 ******* 2019-12-09T09:04:16.730 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2019-12-09T09:04:16.731 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-09T09:04:16.776 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:16 +0000 (0:00:00.050) 0:00:04.058 ******* 2019-12-09T09:04:16.825 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2019-12-09T09:04:16.826 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:16.866 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:16 +0000 (0:00:00.090) 0:00:04.149 ******* 2019-12-09T09:04:17.132 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2019-12-09T09:04:17.132 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-09T09:04:17.205 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:17 +0000 (0:00:00.338) 0:00:04.488 ******* 2019-12-09T09:04:17.234 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2019-12-09T09:04:17.235 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:17.272 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:17 +0000 (0:00:00.067) 0:00:04.555 ******* 2019-12-09T09:04:17.350 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2019-12-09T09:04:17.352 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:04:17.392 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:17 +0000 (0:00:00.119) 0:00:04.675 ******* 2019-12-09T09:04:17.580 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:04:17.617 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:17 +0000 (0:00:00.225) 0:00:04.900 ******* 2019-12-09T09:04:19.002 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2019-12-09T09:04:19.003 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575882257.41-162621457527459/source", "state": "file", "uid": 0} 2019-12-09T09:04:19.051 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:19 +0000 (0:00:01.434) 0:00:06.334 ******* 2019-12-09T09:04:19.069 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575882257.64-133960821659100/source", "state": "file", "uid": 0} 2019-12-09T09:04:19.106 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:19 +0000 (0:00:00.054) 0:00:06.389 ******* 2019-12-09T09:04:19.606 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2019-12-09T09:04:19.607 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:19.628 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:19.939 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:19.940 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:20.254 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:20.265 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:20.562 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:20.563 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:20.875 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:20.908 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:21.168 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:21.264 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:21.473 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:21.602 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:04:21.726 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2019-12-09T09:04:21.732 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2019-12-09T09:04:21.916 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:21 +0000 (0:00:02.809) 0:00:09.198 ******* 2019-12-09T09:04:21.956 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:21 +0000 (0:00:00.040) 0:00:09.239 ******* 2019-12-09T09:04:22.888 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi035.front.sepia.ceph.com] 2019-12-09T09:04:22.990 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2019-12-09T09:04:22.991 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:22 +0000 (0:00:01.034) 0:00:10.273 ******* 2019-12-09T09:04:23.003 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] 2019-12-09T09:04:23.102 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:23 +0000 (0:00:00.112) 0:00:10.385 ******* 2019-12-09T09:04:23.499 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2019-12-09T09:04:23.500 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.225171", "end": "2019-12-09 09:04:23.477046", "rc": 0, "start": "2019-12-09 09:04:23.251875", "stderr": "", "stderr_lines": [], "stdout": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "stdout_lines": ["2c3c6d863a854d6a1e155fc475e8273d88b91491"]} 2019-12-09T09:04:23.540 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:23 +0000 (0:00:00.437) 0:00:10.822 ******* 2019-12-09T09:04:23.557 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.248196", "end": "2019-12-09 09:04:23.521652", "rc": 0, "start": "2019-12-09 09:04:23.273456", "stderr": "", "stderr_lines": [], "stdout": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "stdout_lines": ["2c3c6d863a854d6a1e155fc475e8273d88b91491"]} 2019-12-09T09:04:23.608 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:23 +0000 (0:00:00.068) 0:00:10.891 ******* 2019-12-09T09:04:23.896 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-12-09T09:04:23.896 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.008590", "end": "2019-12-09 09:04:23.868814", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:04:23.860224", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-09T09:04:23.935 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:23 +0000 (0:00:00.327) 0:00:11.218 ******* 2019-12-09T09:04:23.953 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.016280", "end": "2019-12-09 09:04:23.925416", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:04:23.909136", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-09T09:04:23.990 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:23 +0000 (0:00:00.054) 0:00:11.272 ******* 2019-12-09T09:04:23.996 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2019-12-09T09:04:23.997 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:24.034 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.044) 0:00:11.317 ******* 2019-12-09T09:04:24.040 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:24.086 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.046) 0:00:11.364 ******* 2019-12-09T09:04:24.094 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2019-12-09T09:04:24.096 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:24.132 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.050) 0:00:11.415 ******* 2019-12-09T09:04:24.139 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:24.177 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.044) 0:00:11.459 ******* 2019-12-09T09:04:24.182 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-12-09T09:04:24.184 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:24.228 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.051) 0:00:11.511 ******* 2019-12-09T09:04:24.236 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:24.280 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.051) 0:00:11.562 ******* 2019-12-09T09:04:24.285 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2019-12-09T09:04:24.287 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:24.327 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.046) 0:00:11.609 ******* 2019-12-09T09:04:24.335 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:24.373 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.046) 0:00:11.656 ******* 2019-12-09T09:04:24.378 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2019-12-09T09:04:24.380 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:24.421 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.048) 0:00:11.704 ******* 2019-12-09T09:04:24.433 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:04:24.473 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.051) 0:00:11.756 ******* 2019-12-09T09:04:24.746 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2019-12-09T09:04:24.748 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:04:24.793 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.320) 0:00:12.076 ******* 2019-12-09T09:04:24.801 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:04:24.844 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:04:24 +0000 (0:00:00.049) 0:00:12.126 ******* 2019-12-09T09:04:25.154 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-09T09:04:25.154 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-09T09:04:25.242 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-09T09:04:25.476 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-09T09:04:25.574 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-09T09:04:25.805 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-09T09:04:25.950 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-09T09:04:26.149 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-09T09:04:26.288 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-09T09:04:26.467 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-09T09:04:26.640 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-09T09:04:26.806 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-09T09:04:26.984 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-09T09:04:27.136 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-09T09:04:27.332 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-09T09:04:27.458 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-09T09:04:27.687 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-09T09:04:27.806 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-09T09:04:28.039 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-09T09:04:28.135 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-09T09:04:28.395 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-09T09:04:28.477 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-09T09:04:28.737 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-09T09:04:28.816 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-09T09:04:29.100 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-09T09:04:29.155 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-09T09:04:29.451 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-09T09:04:29.474 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-09T09:04:29.784 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-09T09:04:29.828 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-09T09:04:30.145 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-09T09:04:30.157 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-09T09:04:30.494 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-09T09:04:30.507 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-09T09:04:30.828 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-09T09:04:30.841 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-09T09:04:31.171 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-09T09:04:31.173 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-09T09:04:31.514 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-09T09:04:31.528 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-09T09:04:31.851 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-09T09:04:31.897 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-09T09:04:32.193 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-09T09:04:32.247 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-09T09:04:32.534 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-09T09:04:32.593 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-09T09:04:32.889 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-09T09:04:32.946 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-09T09:04:33.232 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-09T09:04:33.234 INFO:teuthology.task.ansible.out: 2019-12-09T09:04:33.319 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-09T09:04:33.587 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-09T09:04:33.669 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-09T09:04:33.942 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-09T09:04:34.007 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-09T09:04:34.297 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-09T09:04:34.340 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-09T09:04:34.672 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-09T09:04:34.674 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-09T09:04:34.999 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-09T09:04:35.035 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-09T09:04:35.327 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-09T09:04:35.393 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-09T09:04:35.666 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-09T09:04:35.742 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-09T09:04:36.030 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-09T09:04:36.064 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-09T09:04:36.409 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-09T09:04:36.411 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-09T09:04:36.739 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-09T09:04:36.793 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-09T09:04:37.087 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-09T09:04:37.140 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-09T09:04:37.424 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-09T09:04:37.499 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-09T09:04:37.774 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-09T09:04:37.871 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-09T09:04:38.121 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-09T09:04:38.247 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-09T09:04:38.468 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:04:38.615 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-09T09:04:38.825 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-09T09:04:38.961 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:04:39.188 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-09T09:04:39.341 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-09T09:04:39.544 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-09T09:04:39.699 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-09T09:04:39.894 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-09T09:04:40.054 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-09T09:04:40.249 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-09T09:04:40.425 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-09T09:04:40.600 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-09T09:04:40.778 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-09T09:04:40.952 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-09T09:04:41.103 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-09T09:04:41.271 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:04:41.452 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-09T09:04:41.611 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-09T09:04:41.793 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:04:41.950 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-09T09:04:42.137 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-09T09:04:42.300 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-09T09:04:42.498 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-09T09:04:42.655 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-09T09:04:42.844 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-09T09:04:42.992 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-09T09:04:43.191 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-09T09:04:43.338 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-09T09:04:43.540 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-09T09:04:43.707 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-09T09:04:43.886 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-09T09:04:44.042 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-09T09:04:44.243 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-09T09:04:44.340 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-09T09:04:44.618 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-09T09:04:44.664 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-09T09:04:44.957 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-09T09:04:45.021 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-09T09:04:45.296 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-09T09:04:45.391 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-09T09:04:45.648 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-09T09:04:45.734 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-09T09:04:45.979 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-09T09:04:46.087 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-09T09:04:46.340 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-09T09:04:46.447 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-09T09:04:46.718 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-09T09:04:46.719 INFO:teuthology.task.ansible.out: 2019-12-09T09:04:46.814 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-09T09:04:47.071 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-09T09:04:47.188 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:04:47.414 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-09T09:04:47.563 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-09T09:04:47.767 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:04:47.909 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-09T09:04:48.123 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-09T09:04:48.251 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-09T09:04:48.485 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-09T09:04:48.602 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-09T09:04:48.825 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-09T09:04:48.983 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-09T09:04:49.150 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-09T09:04:49.318 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-09T09:04:49.508 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-09T09:04:49.668 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-09T09:04:49.848 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-09T09:04:50.043 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-09T09:04:50.194 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-09T09:04:50.400 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-09T09:04:50.529 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-09T09:04:50.755 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-09T09:04:50.885 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-09T09:04:51.108 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-09T09:04:51.259 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-09T09:04:51.434 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-09T09:04:51.596 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-09T09:04:51.597 INFO:teuthology.task.ansible.out: 2019-12-09T09:04:51.788 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-09T09:04:51.949 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-09T09:04:52.134 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-09T09:04:52.306 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-09T09:04:52.473 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-09T09:04:52.633 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-09T09:04:52.823 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-09T09:04:53.012 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-09T09:04:53.171 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-09T09:04:53.377 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-09T09:04:53.495 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-09T09:04:53.763 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-09T09:04:53.857 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-09T09:04:54.101 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-09T09:04:54.212 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-09T09:04:54.449 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-09T09:04:54.588 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-09T09:04:54.798 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-09T09:04:54.916 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-09T09:04:55.203 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-09T09:04:55.257 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-09T09:04:55.585 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-09T09:04:55.625 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-09T09:04:55.929 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-09T09:04:55.974 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-09T09:04:56.308 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-09T09:04:56.345 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-09T09:04:56.661 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-09T09:04:56.663 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-09T09:04:57.028 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-09T09:04:57.044 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-09T09:04:57.410 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-09T09:04:57.432 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-09T09:04:57.770 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-09T09:04:57.805 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-09T09:04:58.140 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-09T09:04:58.183 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-09T09:04:58.491 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-09T09:04:58.547 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-09T09:04:58.854 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-09T09:04:58.904 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-09T09:04:59.221 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-09T09:04:59.286 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-09T09:04:59.559 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-09T09:04:59.655 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-09T09:04:59.898 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-09T09:05:00.033 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-09T09:05:00.232 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-09T09:05:00.405 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-09T09:05:00.587 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:05:00.790 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-09T09:05:00.964 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-09T09:05:01.135 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-09T09:05:01.472 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:05:01.622 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-09T09:05:01.778 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:01 +0000 (0:00:36.935) 0:00:49.061 ******* 2019-12-09T09:05:01.807 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-09T09:05:01.848 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:01 +0000 (0:00:00.069) 0:00:49.131 ******* 2019-12-09T09:05:01.857 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-09T09:05:01.900 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-09T09:05:01.906 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-09T09:05:01.907 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-09T09:05:01.909 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-09T09:05:01.910 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-09T09:05:01.912 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e 2019-12-09T09:05:01.914 INFO:teuthology.task.ansible.out:66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa 2019-12-09T09:05:01.915 INFO:teuthology.task.ansible.out:8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f 2019-12-09T09:05:01.920 INFO:teuthology.task.ansible.out:826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": 2019-12-09T09:05:01.923 INFO:teuthology.task.ansible.out: "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharth 2019-12-09T09:05:01.924 INFO:teuthology.task.ansible.out:anup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c3659458 2019-12-09T09:05:01.926 INFO:teuthology.task.ansible.out:5e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-12-09T09:05:01.999 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:01 +0000 (0:00:00.150) 0:00:49.282 ******* 2019-12-09T09:05:02.576 INFO:teuthology.task.ansible.out: 2019-12-09T09:05:02.577 INFO:teuthology.task.ansible.out:TASK [users : Create all admin users with sudo access.] ************************ changed: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-09T09:05:02.755 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:02 +0000 (0:00:00.755) 0:00:50.038 ******* 2019-12-09T09:05:02.787 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-09T09:05:02.832 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:02 +0000 (0:00:00.076) 0:00:50.114 ******* 2019-12-09T09:05:02.891 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-09T09:05:02.896 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-09T09:05:02.898 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-09T09:05:02.899 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-09T09:05:02.900 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-09T09:05:02.902 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e 2019-12-09T09:05:02.903 INFO:teuthology.task.ansible.out:66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa 2019-12-09T09:05:02.905 INFO:teuthology.task.ansible.out:8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 2019-12-09T09:05:02.916 INFO:teuthology.task.ansible.out:6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f8 2019-12-09T09:05:02.918 INFO:teuthology.task.ansible.out:48846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8a 2019-12-09T09:05:02.919 INFO:teuthology.task.ansible.out:a01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalho 2019-12-09T09:05:02.921 INFO:teuthology.task.ansible.out:jr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-12-09T09:05:02.964 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:02 +0000 (0:00:00.131) 0:00:50.246 ******* 2019-12-09T09:05:03.236 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* ok: [smithi022.front.sepia.ceph.com -> localhost] => {"after": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "attempts": 1, "before": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "changed": false, "remote_url_changed": false} 2019-12-09T09:05:03.283 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:03 +0000 (0:00:00.320) 0:00:50.566 ******* 2019-12-09T09:05:03.691 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:05:03.692 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-09T09:05:03.705 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2019-12-09T09:05:03.706 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com -> localhost] => {"after": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "attempts": 1, "before": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "changed": false, "remote_url_changed": false} 2019-12-09T09:05:03.785 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:03 +0000 (0:00:00.500) 0:00:51.067 ******* 2019-12-09T09:05:04.036 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:05:04.037 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-09T09:05:04.219 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-09T09:05:04.399 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-09T09:05:04.568 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-09T09:05:04.755 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-09T09:05:04.955 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-09T09:05:05.103 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-09T09:05:05.105 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-09T09:05:05.346 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-09T09:05:05.452 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-09T09:05:05.694 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-09T09:05:05.697 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-09T09:05:05.824 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-09T09:05:05.826 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-09T09:05:06.056 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-09T09:05:06.232 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-09T09:05:06.422 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-09T09:05:06.424 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-09T09:05:06.597 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-09T09:05:06.615 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:06.779 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-09T09:05:06.990 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-09T09:05:06.992 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-09T09:05:07.001 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:07.013 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:07.027 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:07.133 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-09T09:05:07.144 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:07.376 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-09T09:05:07.498 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-09T09:05:07.500 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-09T09:05:07.511 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:07.532 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:07.543 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:07.756 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-09T09:05:07.758 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-09T09:05:07.894 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-09T09:05:07.896 INFO:teuthology.task.ansible.out: 2019-12-09T09:05:08.118 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-09T09:05:08.309 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-09T09:05:08.310 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-09T09:05:08.438 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-09T09:05:08.679 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-09T09:05:08.807 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-09T09:05:09.040 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-09T09:05:09.188 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-09T09:05:09.386 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-09T09:05:09.558 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-09T09:05:09.762 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-09T09:05:09.930 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-09T09:05:10.119 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-09T09:05:10.278 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-09T09:05:10.279 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-09T09:05:10.449 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-09T09:05:10.666 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-09T09:05:10.681 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-09T09:05:10.812 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-09T09:05:10.814 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-09T09:05:11.037 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-09T09:05:11.179 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-09T09:05:11.181 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-09T09:05:11.374 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-09T09:05:11.566 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-09T09:05:11.718 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-09T09:05:11.720 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-09T09:05:11.924 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-09T09:05:12.079 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-09T09:05:12.301 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-09T09:05:12.308 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-09T09:05:12.431 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-09T09:05:12.653 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-09T09:05:12.794 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-09T09:05:13.012 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-09T09:05:13.132 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-09T09:05:13.398 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-09T09:05:13.497 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-09T09:05:13.785 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-09T09:05:13.870 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-09T09:05:13.871 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-09T09:05:14.147 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-09T09:05:14.223 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-09T09:05:14.539 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-09T09:05:14.541 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-09T09:05:14.582 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-09T09:05:14.912 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-09T09:05:14.935 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-09T09:05:15.262 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-09T09:05:15.283 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-09T09:05:15.632 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-09T09:05:15.665 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-09T09:05:16.024 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-09T09:05:16.037 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-09T09:05:16.379 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-09T09:05:16.409 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:05:16.734 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-09T09:05:16.767 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-09T09:05:17.119 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:05:17.141 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-09T09:05:17.492 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-09T09:05:17.504 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-09T09:05:17.505 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-09T09:05:17.847 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-09T09:05:17.849 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-09T09:05:17.876 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-09T09:05:18.196 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-09T09:05:18.281 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-09T09:05:18.283 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-09T09:05:18.537 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-09T09:05:18.651 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-09T09:05:18.653 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-09T09:05:18.886 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-09T09:05:19.013 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-09T09:05:19.213 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-09T09:05:19.214 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:05:19.376 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-09T09:05:19.592 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-09T09:05:19.593 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-09T09:05:19.595 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-09T09:05:19.731 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-09T09:05:19.951 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-09T09:05:20.107 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-09T09:05:20.108 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:05:20.311 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-09T09:05:20.462 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-09T09:05:20.464 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-09T09:05:20.465 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-09T09:05:20.654 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-09T09:05:20.833 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-09T09:05:20.990 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-09T09:05:21.184 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-09T09:05:21.330 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-09T09:05:21.535 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-09T09:05:21.685 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-09T09:05:21.919 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-09T09:05:22.060 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-09T09:05:22.267 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-09T09:05:22.414 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-09T09:05:22.586 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-09T09:05:22.810 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-09T09:05:22.811 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-09T09:05:22.917 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-09T09:05:23.161 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-09T09:05:23.277 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-09T09:05:23.505 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-09T09:05:23.654 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-09T09:05:23.656 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-09T09:05:23.858 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-09T09:05:23.859 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-09T09:05:24.020 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-09T09:05:24.205 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-09T09:05:24.396 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-09T09:05:24.581 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-09T09:05:24.759 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-09T09:05:24.761 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-09T09:05:24.949 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-09T09:05:25.130 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-09T09:05:25.324 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:05:25.493 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-09T09:05:25.679 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-09T09:05:25.879 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-09T09:05:26.054 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-09T09:05:26.245 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:05:26.405 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-09T09:05:26.594 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-09T09:05:26.763 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-09T09:05:26.963 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-09T09:05:27.120 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-09T09:05:27.345 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-09T09:05:27.465 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-09T09:05:27.724 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-09T09:05:27.851 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-09T09:05:28.069 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-09T09:05:28.197 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-09T09:05:28.358 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-09T09:05:28.362 INFO:teuthology.task.ansible.out: 2019-12-09T09:05:28.542 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-09T09:05:28.701 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-09T09:05:28.869 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-09T09:05:29.071 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-09T09:05:29.208 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-09T09:05:29.424 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-09T09:05:29.585 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-09T09:05:29.774 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-09T09:05:29.959 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9V 2019-12-09T09:05:29.961 INFO:teuthology.task.ansible.out:ENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-09T09:05:30.140 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-09T09:05:30.293 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-09T09:05:30.485 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-09T09:05:30.637 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-09T09:05:30.813 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-09T09:05:30.999 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-09T09:05:31.192 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-09T09:05:31.337 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-09T09:05:31.556 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-09T09:05:31.681 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-09T09:05:31.924 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-09T09:05:32.018 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-09T09:05:32.287 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-09T09:05:32.361 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-09T09:05:32.673 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-09T09:05:32.679 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-09T09:05:32.988 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-09T09:05:33.021 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-09T09:05:33.023 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-09T09:05:33.386 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-09T09:05:33.387 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-09T09:05:33.389 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-09T09:05:33.717 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-09T09:05:33.773 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-09T09:05:34.052 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-09T09:05:34.139 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-09T09:05:34.140 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-09T09:05:34.422 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-09T09:05:34.513 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-09T09:05:34.515 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-09T09:05:34.758 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-09T09:05:34.896 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-09T09:05:35.132 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-09T09:05:35.249 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-09T09:05:35.496 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-09T09:05:35.608 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-09T09:05:35.835 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-09T09:05:35.954 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-09T09:05:36.199 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-09T09:05:36.314 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-09T09:05:36.558 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-09T09:05:36.665 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-09T09:05:36.924 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-09T09:05:37.019 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-09T09:05:37.261 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-09T09:05:37.398 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-09T09:05:37.601 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-12-09T09:05:37.774 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-09T09:05:37.934 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-09T09:05:38.141 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-09T09:05:38.293 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-09T09:05:38.464 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-09T09:05:38.656 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-12-09T09:05:38.658 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:05:38.805 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-12-09T09:05:39.004 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-09T09:05:39.159 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-09T09:05:39.368 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-09T09:05:39.507 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:39 +0000 (0:00:35.722) 0:01:26.790 ******* 2019-12-09T09:05:39.515 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-09T09:05:39.566 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:39.579 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:39.590 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:39.608 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d2 2019-12-09T09:05:39.610 INFO:teuthology.task.ansible.out:49450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:39.626 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:39.627 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:39.638 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:39.862 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsri 2019-12-09T09:05:39.883 INFO:teuthology.task.ansible.out:vas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:05:39.988 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:05:39.991 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-09T09:05:40.000 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:40.210 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:05:40.212 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-09T09:05:40.338 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-09T09:05:40.561 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:05:40.562 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-09T09:05:40.687 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:40 +0000 (0:00:01.179) 0:01:27.969 ******* 2019-12-09T09:05:40.693 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:05:40.694 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-09T09:05:40.734 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:40.746 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:40.757 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:40.758 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:40.769 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:40.781 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:40.793 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:40.804 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:40.815 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.073 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-09T09:05:41.090 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.096 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.117 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.129 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.147 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.160 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.164 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.174 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.186 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-09T09:05:41.189 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.189 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.201 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.213 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.224 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.235 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.247 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.258 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.259 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.281 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.282 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.293 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.304 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.315 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.333 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.348 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.359 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.371 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.382 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.393 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.394 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.405 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.417 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.428 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.440 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.442 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.453 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.467 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.477 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.489 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.500 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.512 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.523 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.524 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-09T09:05:41.535 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.546 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.568 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.579 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.581 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.591 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.603 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.603 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.614 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.626 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.637 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.638 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.650 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.660 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.677 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.683 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.694 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.695 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.706 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.717 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.728 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.740 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.741 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.752 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.763 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.774 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.775 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.786 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.808 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.820 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.821 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.822 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.835 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.844 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.845 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.856 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.867 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.889 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-09T09:05:41.891 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.901 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.902 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.918 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.924 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.936 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.947 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.948 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.959 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.978 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:41.985 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.034 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:42 +0000 (0:00:01.347) 0:01:29.317 ******* 2019-12-09T09:05:42.072 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2019-12-09T09:05:42.074 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.109 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:42 +0000 (0:00:00.075) 0:01:29.392 ******* 2019-12-09T09:05:42.215 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-09T09:05:42.237 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.247 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.257 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.279 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.290 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.302 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.315 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.318 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.330 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.341 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.352 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.364 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.397 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.399 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.400 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.410 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.422 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.433 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.434 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.445 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.456 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.467 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.478 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.479 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.492 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi022.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.503 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.513 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.529 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.531 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.544 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.555 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.557 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.569 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.580 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.591 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.617 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.618 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.619 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.630 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.651 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.657 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.664 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.674 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.686 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.710 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.711 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.713 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.725 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.735 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.759 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.760 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.761 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.773 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.785 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.786 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.797 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.808 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.826 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.835 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.845 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.846 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:05:42.852 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.861 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.869 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.880 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.882 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.892 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.903 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.915 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.916 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.927 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.938 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.950 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.961 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.973 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.985 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.986 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:42.998 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:43.011 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:43.022 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:43.023 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:43.035 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:43.047 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:43.048 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:43.071 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:43.085 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} skipping: [smithi035.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:43.169 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:43 +0000 (0:00:01.059) 0:01:30.452 ******* 2019-12-09T09:05:43.197 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2019-12-09T09:05:43.199 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:05:43.242 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:43 +0000 (0:00:00.073) 0:01:30.525 ******* 2019-12-09T09:05:43.249 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:05:43.249 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-09T09:05:43.566 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-09T09:05:43.631 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-09T09:05:43.926 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-12-09T09:05:43.982 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-09T09:05:44.298 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-09T09:05:44.300 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-09T09:05:44.649 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-09T09:05:44.651 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-09T09:05:44.984 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-12-09T09:05:45.000 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-09T09:05:45.317 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-09T09:05:45.349 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-09T09:05:45.641 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-09T09:05:45.705 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-09T09:05:45.987 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-09T09:05:46.029 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-09T09:05:46.308 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-09T09:05:46.371 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-09T09:05:46.653 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-09T09:05:46.727 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-09T09:05:46.989 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-09T09:05:47.107 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-09T09:05:47.109 INFO:teuthology.task.ansible.out: 2019-12-09T09:05:47.341 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-09T09:05:47.342 INFO:teuthology.task.ansible.out: 2019-12-09T09:05:47.448 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-09T09:05:47.449 INFO:teuthology.task.ansible.out: 2019-12-09T09:05:47.669 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-09T09:05:47.671 INFO:teuthology.task.ansible.out: 2019-12-09T09:05:47.786 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-09T09:05:47.786 INFO:teuthology.task.ansible.out: 2019-12-09T09:05:47.977 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-09T09:05:48.131 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-09T09:05:48.288 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-09T09:05:48.489 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-09T09:05:48.614 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-09T09:05:48.853 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-09T09:05:48.909 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-09T09:05:49.221 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-09T09:05:49.244 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-09T09:05:49.570 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-09T09:05:49.601 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-09T09:05:49.942 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-09T09:05:50.167 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-09T09:05:50.169 INFO:teuthology.task.ansible.out: 2019-12-09T09:05:50.287 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-09T09:05:50.698 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-09T09:05:50.878 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-09T09:05:51.262 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-09T09:05:51.481 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-09T09:05:51.964 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-09T09:05:52.233 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-09T09:05:52.547 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-09T09:05:52.816 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-09T09:05:53.059 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-09T09:05:53.507 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-09T09:05:53.665 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-09T09:05:54.296 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-09T09:05:54.370 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-09T09:05:54.892 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-09T09:05:54.945 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-09T09:05:55.441 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-09T09:05:55.495 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-09T09:05:56.007 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-09T09:05:56.082 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-09T09:05:56.622 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-09T09:05:56.676 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-09T09:05:57.039 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-09T09:05:57.237 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-09T09:05:57.385 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-09T09:05:57.719 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-09T09:05:57.833 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-09T09:05:58.065 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-09T09:05:58.153 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-09T09:05:58.393 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-09T09:05:58.476 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-09T09:05:58.721 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-09T09:05:58.786 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-09T09:05:59.069 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-09T09:05:59.102 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-09T09:05:59.453 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-09T09:05:59.595 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-09T09:05:59.653 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:05:59 +0000 (0:00:16.410) 0:01:46.935 ******* 2019-12-09T09:05:59.771 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-09T09:06:00.105 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-09T09:06:00.757 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-09T09:06:00.847 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:00 +0000 (0:00:01.193) 0:01:48.129 ******* 2019-12-09T09:06:00.874 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-09T09:06:00.876 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "checksum": "8a303ebcf60c08d57f11c57c9682a02d84685d0b", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c2805c19b557daad4717860e2c7029bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575882359.69-144732762103587/source", "state": "file", "uid": 0} 2019-12-09T09:06:00.913 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:00 +0000 (0:00:00.066) 0:01:48.196 ******* 2019-12-09T09:06:00.953 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2019-12-09T09:06:00.955 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => { "msg": "Host smithi022.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-09T09:06:00.993 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:00 +0000 (0:00:00.079) 0:01:48.275 ******* 2019-12-09T09:06:01.030 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2019-12-09T09:06:01.032 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-09T09:06:01.075 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:01 +0000 (0:00:00.081) 0:01:48.357 ******* 2019-12-09T09:06:01.112 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-12-09T09:06:01.114 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:01.154 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:01 +0000 (0:00:00.079) 0:01:48.436 ******* 2019-12-09T09:06:01.591 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2019-12-09T09:06:01.593 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.103434", "end": "2019-12-09 09:06:01.555863", "rc": 0, "start": "2019-12-09 09:06:01.452429", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-09T09:06:01.639 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:01 +0000 (0:00:00.484) 0:01:48.921 ******* 2019-12-09T09:06:01.667 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-12-09T09:06:01.669 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:01.709 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:01 +0000 (0:00:00.069) 0:01:48.991 ******* 2019-12-09T09:06:01.736 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-12-09T09:06:01.738 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:01.799 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:01 +0000 (0:00:00.088) 0:01:49.080 ******* 2019-12-09T09:06:01.825 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-09T09:06:01.827 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:01.882 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:01 +0000 (0:00:00.084) 0:01:49.164 ******* 2019-12-09T09:06:01.921 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:01.961 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:01 +0000 (0:00:00.079) 0:01:49.244 ******* 2019-12-09T09:06:01.989 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:02.033 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.071) 0:01:49.316 ******* 2019-12-09T09:06:02.040 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-09T09:06:02.042 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "checksum": "8a303ebcf60c08d57f11c57c9682a02d84685d0b", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c2805c19b557daad4717860e2c7029bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575882360.87-249365318220358/source", "state": "file", "uid": 0} 2019-12-09T09:06:02.091 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.057) 0:01:49.373 ******* 2019-12-09T09:06:02.096 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-12-09T09:06:02.097 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:02.136 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.044) 0:01:49.418 ******* 2019-12-09T09:06:02.142 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [smithi035.front.sepia.ceph.com] => { "msg": "Host smithi035.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-09T09:06:02.185 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.046) 0:01:49.465 ******* 2019-12-09T09:06:02.188 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-12-09T09:06:02.190 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi022.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:06:02.230 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.047) 0:01:49.513 ******* 2019-12-09T09:06:02.236 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2019-12-09T09:06:02.237 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-09T09:06:02.277 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.047) 0:01:49.560 ******* 2019-12-09T09:06:02.283 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-12-09T09:06:02.285 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:02.325 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.047) 0:01:49.608 ******* 2019-12-09T09:06:02.332 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-12-09T09:06:02.333 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:02.375 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.050) 0:01:49.658 ******* 2019-12-09T09:06:02.382 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-12-09T09:06:02.383 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:02.421 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.045) 0:01:49.703 ******* 2019-12-09T09:06:02.462 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-12-09T09:06:02.464 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:06:02.500 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.078) 0:01:49.782 ******* 2019-12-09T09:06:02.540 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-12-09T09:06:02.541 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:02.577 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.077) 0:01:49.860 ******* 2019-12-09T09:06:02.616 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-12-09T09:06:02.617 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:06:02.659 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.082) 0:01:49.942 ******* 2019-12-09T09:06:02.719 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-09T09:06:02.732 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:02.779 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.119) 0:01:50.062 ******* 2019-12-09T09:06:02.807 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-12-09T09:06:02.809 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:02.850 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.071) 0:01:50.133 ******* 2019-12-09T09:06:02.857 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2019-12-09T09:06:02.858 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.108429", "end": "2019-12-09 09:06:02.790550", "rc": 0, "start": "2019-12-09 09:06:02.682121", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-09T09:06:02.903 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.052) 0:01:50.185 ******* 2019-12-09T09:06:02.909 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-12-09T09:06:02.911 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:02.950 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.047) 0:01:50.233 ******* 2019-12-09T09:06:02.959 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:02.998 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:02 +0000 (0:00:00.047) 0:01:50.281 ******* 2019-12-09T09:06:03.004 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-09T09:06:03.008 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.049 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.050) 0:01:50.331 ******* 2019-12-09T09:06:03.057 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-12-09T09:06:03.059 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.112 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.063) 0:01:50.395 ******* 2019-12-09T09:06:03.119 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-12-09T09:06:03.121 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:06:03.168 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.055) 0:01:50.450 ******* 2019-12-09T09:06:03.175 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.225 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.057) 0:01:50.508 ******* 2019-12-09T09:06:03.231 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-09T09:06:03.234 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.276 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.050) 0:01:50.558 ******* 2019-12-09T09:06:03.282 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-12-09T09:06:03.283 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.321 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.044) 0:01:50.603 ******* 2019-12-09T09:06:03.326 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-12-09T09:06:03.328 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.365 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.044) 0:01:50.648 ******* 2019-12-09T09:06:03.372 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-12-09T09:06:03.374 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.417 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.045) 0:01:50.694 ******* TASK [common : Store list of enabled repos] ************************************ skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.456 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.045) 0:01:50.739 ******* 2019-12-09T09:06:03.465 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-12-09T09:06:03.467 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.502 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.045) 0:01:50.785 ******* 2019-12-09T09:06:03.508 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2019-12-09T09:06:03.510 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.551 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.048) 0:01:50.833 ******* 2019-12-09T09:06:03.557 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-12-09T09:06:03.558 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:06:03.560 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:06:03.610 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.059) 0:01:50.892 ******* 2019-12-09T09:06:03.616 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-12-09T09:06:03.618 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.659 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.048) 0:01:50.941 ******* 2019-12-09T09:06:03.669 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-12-09T09:06:03.670 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.714 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.055) 0:01:50.997 ******* 2019-12-09T09:06:03.720 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-12-09T09:06:03.722 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.768 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.053) 0:01:51.051 ******* 2019-12-09T09:06:03.775 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:03.813 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.045) 0:01:51.096 ******* 2019-12-09T09:06:03.820 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-12-09T09:06:03.822 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:06:03.862 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.048) 0:01:51.145 ******* 2019-12-09T09:06:03.874 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-12-09T09:06:03.875 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:06:03.921 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.058) 0:01:51.204 ******* 2019-12-09T09:06:03.930 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-09T09:06:03.972 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:03 +0000 (0:00:00.050) 0:01:51.255 ******* 2019-12-09T09:06:03.979 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-12-09T09:06:03.981 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:04.021 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.049) 0:01:51.304 ******* 2019-12-09T09:06:04.028 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-12-09T09:06:04.028 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:04.075 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.053) 0:01:51.357 ******* 2019-12-09T09:06:04.081 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-12-09T09:06:04.082 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:06:04.126 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.049) 0:01:51.407 ******* 2019-12-09T09:06:04.161 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-09T09:06:04.163 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:04.205 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.080) 0:01:51.488 ******* 2019-12-09T09:06:04.243 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-12-09T09:06:04.246 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:04.286 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.080) 0:01:51.568 ******* 2019-12-09T09:06:04.330 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-12-09T09:06:04.331 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:04.369 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.083) 0:01:51.652 ******* 2019-12-09T09:06:04.396 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-09T09:06:04.398 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:04.436 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.066) 0:01:51.719 ******* 2019-12-09T09:06:04.452 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-12-09T09:06:04.453 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:06:04.491 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.054) 0:01:51.774 ******* 2019-12-09T09:06:04.497 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-12-09T09:06:04.499 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:06:04.537 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.045) 0:01:51.820 ******* 2019-12-09T09:06:04.584 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-09T09:06:04.586 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:04.628 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.091) 0:01:51.911 ******* 2019-12-09T09:06:04.656 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:04.721 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.092) 0:01:52.004 ******* 2019-12-09T09:06:04.722 INFO:teuthology.task.ansible.out: 2019-12-09T09:06:04.751 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2019-12-09T09:06:04.753 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:04.802 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.080) 0:01:52.085 ******* 2019-12-09T09:06:04.829 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2019-12-09T09:06:04.831 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:04.878 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.076) 0:01:52.161 ******* 2019-12-09T09:06:04.908 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:04.948 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:04 +0000 (0:00:00.069) 0:01:52.231 ******* 2019-12-09T09:06:04.987 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-12-09T09:06:04.996 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:05.045 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:05 +0000 (0:00:00.097) 0:01:52.328 ******* 2019-12-09T09:06:05.083 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-12-09T09:06:05.085 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:06:05.124 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:05 +0000 (0:00:00.078) 0:01:52.407 ******* 2019-12-09T09:06:05.152 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-09T09:06:05.185 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:05 +0000 (0:00:00.060) 0:01:52.467 ******* 2019-12-09T09:06:05.222 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-12-09T09:06:05.224 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:05.262 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:05 +0000 (0:00:00.077) 0:01:52.544 ******* 2019-12-09T09:06:05.633 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** ok: [smithi035.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:06:05.670 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:05 +0000 (0:00:00.407) 0:01:52.952 ******* 2019-12-09T09:06:05.818 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-09T09:06:05.819 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-09T09:06:06.737 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-09T09:06:06.751 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-09T09:06:06.826 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:06 +0000 (0:00:01.156) 0:01:54.109 ******* 2019-12-09T09:06:07.723 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-09T09:06:07.780 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:07 +0000 (0:00:00.954) 0:01:55.063 ******* 2019-12-09T09:06:07.963 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-09T09:06:07.964 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575882366.87-154218552226096/source", "state": "file", "uid": 0} 2019-12-09T09:06:08.900 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575882367.83-57426233066250/source", "state": "file", "uid": 0} 2019-12-09T09:06:08.903 INFO:teuthology.task.ansible.out: 2019-12-09T09:06:09.155 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575882367.97-136394058146556/source", "state": "file", "uid": 0} 2019-12-09T09:06:09.200 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:09 +0000 (0:00:01.419) 0:01:56.483 ******* 2019-12-09T09:06:09.849 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-12-09T09:06:09.850 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-09T09:06:10.018 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-09T09:06:10.019 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575882368.92-215960419488684/source", "state": "file", "uid": 0} 2019-12-09T09:06:10.064 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:10 +0000 (0:00:00.863) 0:01:57.347 ******* 2019-12-09T09:06:10.180 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:06:10.182 INFO:teuthology.task.ansible.out: [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually 2019-12-09T09:06:10.236 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:10 +0000 (0:00:00.172) 0:01:57.519 ******* 2019-12-09T09:06:10.440 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-09T09:06:10.756 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:06:10.824 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:10 +0000 (0:00:00.586) 0:01:58.106 ******* 2019-12-09T09:06:23.146 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** [WARNING]: Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:11.958170", "end": "2019-12-09 09:06:23.115361", "rc": 0, "start": "2019-12-09 09:06:11.157191", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: repos-va.psychz.net\n * extras: mirror.math.princeton.edu\n * updates: mirror.cogentco.com\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: repos-va.psychz.net", " * extras: mirror.math.princeton.edu", " * updates: mirror.cogentco.com", "Metadata Cache Created"]} 2019-12-09T09:06:23.184 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:23 +0000 (0:00:12.360) 0:02:10.467 ******* 2019-12-09T09:06:23.266 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.669686", "end": "2019-12-09 09:06:23.225374", "rc": 0, "start": "2019-12-09 09:06:10.555688", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: repos-va.psychz.net\n * extras: mirror.math.princeton.edu\n * updates: mirror.cogentco.com\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: repos-va.psychz.net", " * extras: mirror.math.princeton.edu", " * updates: mirror.cogentco.com", "Metadata Cache Created"]} 2019-12-09T09:06:23.322 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:23 +0000 (0:00:00.138) 0:02:10.605 ******* 2019-12-09T09:06:23.549 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-12-09T09:06:23.549 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-09T09:06:23.700 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-09T09:06:23.863 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:06:23.902 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:23 +0000 (0:00:00.579) 0:02:11.184 ******* 2019-12-09T09:06:23.950 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T09:06:23.951 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:23.997 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:23 +0000 (0:00:00.095) 0:02:11.280 ******* 2019-12-09T09:06:24.027 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-12-09T09:06:24.028 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.030 INFO:teuthology.task.ansible.out: 2019-12-09T09:06:24.032 INFO:teuthology.task.ansible.out:TASK [common : Uncomment mirrorlist] ******************************************* 2019-12-09T09:06:24.033 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:06:24.090 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.093) 0:02:11.373 ******* 2019-12-09T09:06:24.133 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.042) 0:02:11.416 ******* 2019-12-09T09:06:24.152 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.205 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.071) 0:02:11.487 ******* 2019-12-09T09:06:24.211 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-12-09T09:06:24.213 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.256 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.051) 0:02:11.538 ******* 2019-12-09T09:06:24.263 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-12-09T09:06:24.264 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.316 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.059) 0:02:11.598 ******* 2019-12-09T09:06:24.322 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T09:06:24.324 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.367 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.051) 0:02:11.650 ******* 2019-12-09T09:06:24.373 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-12-09T09:06:24.375 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.412 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.044) 0:02:11.694 ******* 2019-12-09T09:06:24.418 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-12-09T09:06:24.420 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.476 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.064) 0:02:11.759 ******* 2019-12-09T09:06:24.486 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T09:06:24.487 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.529 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.053) 0:02:11.812 ******* 2019-12-09T09:06:24.535 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-12-09T09:06:24.537 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.591 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.061) 0:02:11.874 ******* 2019-12-09T09:06:24.597 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-12-09T09:06:24.599 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.654 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.062) 0:02:11.937 ******* 2019-12-09T09:06:24.663 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.709 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.055) 0:02:11.992 ******* 2019-12-09T09:06:24.717 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-12-09T09:06:24.719 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.753 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.043) 0:02:12.036 ******* 2019-12-09T09:06:24.759 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-12-09T09:06:24.761 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.800 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.047) 0:02:12.083 ******* 2019-12-09T09:06:24.807 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2019-12-09T09:06:24.808 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.867 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.066) 0:02:12.150 ******* 2019-12-09T09:06:24.896 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-12-09T09:06:24.897 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:24.935 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:24 +0000 (0:00:00.067) 0:02:12.218 ******* 2019-12-09T09:06:25.532 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.392788", "end": "2019-12-09 09:06:25.500587", "rc": 0, "start": "2019-12-09 09:06:25.107799", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:06:25.578 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:25 +0000 (0:00:00.643) 0:02:12.861 ******* 2019-12-09T09:06:26.454 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-12-09T09:06:26.455 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-09T09:06:26.521 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:26 +0000 (0:00:00.942) 0:02:13.804 ******* 2019-12-09T09:06:26.537 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-12-09T09:06:26.538 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.262187", "end": "2019-12-09 09:06:26.500463", "rc": 0, "start": "2019-12-09 09:06:25.238276", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:06:26.580 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:26 +0000 (0:00:00.058) 0:02:13.863 ******* 2019-12-09T09:06:26.586 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-09T09:06:26.587 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:26.645 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:26 +0000 (0:00:00.065) 0:02:13.928 ******* 2019-12-09T09:06:26.678 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-09T09:06:26.679 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:26.736 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:26 +0000 (0:00:00.090) 0:02:14.018 ******* 2019-12-09T09:06:26.774 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-09T09:06:26.775 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:26.828 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:26 +0000 (0:00:00.092) 0:02:14.111 ******* 2019-12-09T09:06:26.856 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-09T09:06:26.857 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:26.920 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:26 +0000 (0:00:00.091) 0:02:14.203 ******* 2019-12-09T09:06:27.547 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-12-09T09:06:27.548 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-09T09:06:27.592 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:27 +0000 (0:00:00.672) 0:02:14.875 ******* 2019-12-09T09:06:27.641 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-09T09:06:27.643 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:27.690 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:27 +0000 (0:00:00.097) 0:02:14.973 ******* 2019-12-09T09:06:27.718 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-09T09:06:27.720 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:27.780 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:27 +0000 (0:00:00.088) 0:02:15.062 ******* 2019-12-09T09:06:27.808 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-09T09:06:27.810 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:27.871 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:27 +0000 (0:00:00.092) 0:02:15.154 ******* 2019-12-09T09:06:27.903 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2019-12-09T09:06:27.905 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-09T09:06:27.906 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-09T09:06:27.907 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:27.962 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:27 +0000 (0:00:00.090) 0:02:15.244 ******* 2019-12-09T09:06:28.010 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:28 +0000 (0:00:00.048) 0:02:15.293 ******* 2019-12-09T09:06:28.942 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2019-12-09T09:06:28.944 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-09T09:06:28.995 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:28 +0000 (0:00:00.985) 0:02:16.278 ******* 2019-12-09T09:06:32.008 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2019-12-09T09:06:32.014 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-09T09:06:32.056 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:32 +0000 (0:00:03.054) 0:02:19.333 ******* 2019-12-09T09:06:32.902 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-09T09:06:32.982 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:32 +0000 (0:00:00.931) 0:02:20.265 ******* 2019-12-09T09:06:33.066 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-09T09:06:33.067 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-09T09:06:34.015 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-09T09:06:34.021 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-09T09:06:34.949 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-09T09:06:34.976 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-09T09:06:35.018 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:35 +0000 (0:00:02.035) 0:02:22.301 ******* 2019-12-09T09:06:35.019 INFO:teuthology.task.ansible.out: 2019-12-09T09:06:35.347 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2019-12-09T09:06:35.348 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:06:35.387 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:35 +0000 (0:00:00.368) 0:02:22.669 ******* 2019-12-09T09:06:35.904 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-09T09:06:35.907 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-09T09:06:35.964 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:35 +0000 (0:00:00.577) 0:02:23.246 ******* 2019-12-09T09:06:36.299 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:06:36.344 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:36 +0000 (0:00:00.380) 0:02:23.627 ******* 2019-12-09T09:06:36.352 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-09T09:06:36.353 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-09T09:06:37.253 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-09T09:06:37.290 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-09T09:06:38.168 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-09T09:06:38.169 INFO:teuthology.task.ansible.out: 2019-12-09T09:06:38.184 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-09T09:06:38.186 INFO:teuthology.task.ansible.out: 2019-12-09T09:06:38.236 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:38 +0000 (0:00:01.892) 0:02:25.519 ******* 2019-12-09T09:06:38.264 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-12-09T09:06:38.266 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:38.318 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:38 +0000 (0:00:00.081) 0:02:25.601 ******* 2019-12-09T09:06:38.355 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-12-09T09:06:38.356 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:06:38.393 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:38 +0000 (0:00:00.074) 0:02:25.675 ******* 2019-12-09T09:06:38.738 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-12-09T09:06:38.742 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-09T09:06:38.786 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:38 +0000 (0:00:00.393) 0:02:26.069 ******* 2019-12-09T09:06:38.838 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-12-09T09:06:38.839 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:06:38.849 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:06:38.860 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:06:38.873 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:06:38.920 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:38 +0000 (0:00:00.134) 0:02:26.203 ******* 2019-12-09T09:06:39.109 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi022.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-09T09:06:39.147 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:39 +0000 (0:00:00.226) 0:02:26.430 ******* 2019-12-09T09:06:39.149 INFO:teuthology.task.ansible.out: 2019-12-09T09:06:39.174 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-12-09T09:06:39.176 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:39.212 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:39 +0000 (0:00:00.064) 0:02:26.495 ******* 2019-12-09T09:06:39.260 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-12-09T09:06:39.260 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:06:39.298 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:39 +0000 (0:00:00.086) 0:02:26.581 ******* 2019-12-09T09:06:39.640 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-12-09T09:06:39.642 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-09T09:06:39.679 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:39 +0000 (0:00:00.380) 0:02:26.961 ******* 2019-12-09T09:06:39.718 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-12-09T09:06:39.719 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:06:39.739 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:06:39.764 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:06:39.784 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:06:39.827 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:39 +0000 (0:00:00.148) 0:02:27.110 ******* 2019-12-09T09:06:42.421 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-12-09T09:06:42.422 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-09T09:06:42.468 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:42 +0000 (0:00:02.640) 0:02:29.750 ******* 2019-12-09T09:06:42.506 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-09T09:06:42.508 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:42.549 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:42 +0000 (0:00:00.080) 0:02:29.831 ******* 2019-12-09T09:06:43.167 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-12-09T09:06:43.169 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-09T09:06:43.230 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:43 +0000 (0:00:00.680) 0:02:30.512 ******* 2019-12-09T09:06:43.260 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:43.304 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:43 +0000 (0:00:00.074) 0:02:30.587 ******* 2019-12-09T09:06:43.618 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2019-12-09T09:06:43.619 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-09T09:06:43.681 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:43 +0000 (0:00:00.376) 0:02:30.964 ******* 2019-12-09T09:06:44.228 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-09T09:06:44.303 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:44 +0000 (0:00:00.621) 0:02:31.586 ******* 2019-12-09T09:06:44.628 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2019-12-09T09:06:44.629 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-09T09:06:44.726 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:44 +0000 (0:00:00.423) 0:02:32.009 ******* 2019-12-09T09:06:44.766 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2019-12-09T09:06:44.767 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:44.803 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:44 +0000 (0:00:00.076) 0:02:32.086 ******* 2019-12-09T09:06:45.122 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-12-09T09:06:45.123 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:06:45.179 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:45 +0000 (0:00:00.373) 0:02:32.459 ******* 2019-12-09T09:06:45.265 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2019-12-09T09:06:45.266 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-09T09:06:45.304 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:45 +0000 (0:00:00.127) 0:02:32.587 ******* 2019-12-09T09:06:45.333 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2019-12-09T09:06:45.334 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:45.373 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:45 +0000 (0:00:00.068) 0:02:32.656 ******* 2019-12-09T09:06:45.577 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-12-09T09:06:45.579 INFO:teuthology.task.ansible.out:fatal: [smithi035.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-09T09:06:45.729 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:45 +0000 (0:00:00.356) 0:02:33.012 ******* 2019-12-09T09:06:46.365 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-12-09T09:06:46.367 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:46.417 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:46 +0000 (0:00:00.687) 0:02:33.700 ******* 2019-12-09T09:06:46.734 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-12-09T09:06:46.735 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:06:46.779 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:46 +0000 (0:00:00.362) 0:02:34.062 ******* 2019-12-09T09:06:47.148 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-12-09T09:06:47.149 INFO:teuthology.task.ansible.out:fatal: [smithi022.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-09T09:06:47.186 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:47 +0000 (0:00:00.406) 0:02:34.468 ******* 2019-12-09T09:06:47.222 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-12-09T09:06:47.224 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:06:47.261 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:47 +0000 (0:00:00.075) 0:02:34.544 ******* 2019-12-09T09:06:47.671 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2019-12-09T09:06:47.672 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-09T09:06:47.712 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:47 +0000 (0:00:00.450) 0:02:34.995 ******* 2019-12-09T09:06:48.252 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-09T09:06:48.292 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:48 +0000 (0:00:00.579) 0:02:35.574 ******* 2019-12-09T09:06:48.556 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2019-12-09T09:06:48.557 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2019-12-09 09:03:36 UTC", "ActiveEnterTimestampMonotonic": "21353723", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "local-fs.target network.target system.slice systemd-journald.socket var-run.mount nss-lookup.target basic.target time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Mon 2019-12-09 09:03:36 UTC", "AssertTimestampMonotonic": "21353203", "Before": "shutdown.target multi-user.target getty@tty1.service xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2019-12-09 09:03:36 UTC", "ConditionTimestampMonotonic": "21353201", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8743", "ExecMainStartTimestamp": "Mon 2019-12-09 09:03:36 UTC", "ExecMainStartTimestampMonotonic": "21353660", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-12-09T09:06:48.558 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Mon 2019-12-09 09:03:36 UTC] ; stop_time=[n/a] ; pid=8743 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2019-12-09 09:03:36 UTC", "InactiveExitTimestampMonotonic": "21353723", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8743", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-12-09T09:06:48.560 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Mon 2019-12-09 09:03:36 UTC", "WatchdogTimestampMonotonic": "21353698", "WatchdogUSec": "0"}} 2019-12-09T09:06:48.599 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:48 +0000 (0:00:00.306) 0:02:35.881 ******* 2019-12-09T09:06:48.762 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2019-12-09 09:03:37 UTC", "ActiveEnterTimestampMonotonic": "18960932", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "var-run.mount systemd-journald.socket nss-lookup.target time-sync.target system.slice local-fs.target basic.target network.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Mon 2019-12-09 09:03:37 UTC", "AssertTimestampMonotonic": "18960103", "Before": "xdm.service getty@tty1.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2019-12-09 09:03:37 UTC", "ConditionTimestampMonotonic": "18960103", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8786", "ExecMainStartTimestamp": "Mon 2019-12-09 09:03:37 UTC", "ExecMainStartTimestampMonotonic": "18960854", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-12-09T09:06:48.764 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Mon 2019-12-09 09:03:37 UTC] ; stop_time=[n/a] ; pid=8786 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2019-12-09 09:03:37 UTC", "InactiveExitTimestampMonotonic": "18960932", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8786", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target network-online.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-12-09T09:06:48.765 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Mon 2019-12-09 09:03:37 UTC", "WatchdogTimestampMonotonic": "18960901", "WatchdogUSec": "0"}} 2019-12-09T09:06:48.811 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:48 +0000 (0:00:00.212) 0:02:36.094 ******* 2019-12-09T09:06:48.940 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2019-12-09T09:06:48.942 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004271", "end": "2019-12-09 09:06:48.914730", "rc": 0, "start": "2019-12-09 09:06:48.910459", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-09T09:06:48.982 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:48 +0000 (0:00:00.170) 0:02:36.264 ******* 2019-12-09T09:06:49.167 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004412", "end": "2019-12-09 09:06:49.138371", "rc": 0, "start": "2019-12-09 09:06:49.133959", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-09T09:06:49.214 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:49 +0000 (0:00:00.232) 0:02:36.497 ******* 2019-12-09T09:06:49.469 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2019-12-09T09:06:49.472 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-09T09:06:49.520 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:49 +0000 (0:00:00.305) 0:02:36.802 ******* 2019-12-09T09:06:49.699 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-09T09:06:49.759 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:06:49 +0000 (0:00:00.239) 0:02:37.042 ******* 2019-12-09T09:07:17.145 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: repos-va.psychz.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.math.princeton.edu\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n---------------------------------- 2019-12-09T09:07:17.147 INFO:teuthology.task.ansible.out:----------------------------------------------\nTotal 23 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-09T09:07:17.192 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:17 +0000 (0:00:27.428) 0:03:04.470 ******* 2019-12-09T09:07:17.507 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: repos-va.psychz.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.math.princeton.edu\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 2019-12-09T09:07:17.509 INFO:teuthology.task.ansible.out: 24 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-09T09:07:17.547 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:17 +0000 (0:00:00.359) 0:03:04.830 ******* 2019-12-09T09:07:17.936 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-09T09:07:17.971 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:17 +0000 (0:00:00.424) 0:03:05.254 ******* 2019-12-09T09:07:18.108 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-09T09:07:18.153 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:18 +0000 (0:00:00.181) 0:03:05.436 ******* 2019-12-09T09:07:26.881 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:08.546914", "end": "2019-12-09 09:07:26.848535", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:07:18.301621", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-09T09:07:26.882 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:08.399763", "end": "2019-12-09 09:07:26.845938", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:07:18.446175", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-09T09:07:26.943 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:26 +0000 (0:00:08.789) 0:03:14.225 ******* 2019-12-09T09:07:26.982 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:26 +0000 (0:00:00.038) 0:03:14.264 ******* 2019-12-09T09:07:27.967 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2019-12-09T09:07:27.968 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-09T09:07:28.014 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:28 +0000 (0:00:01.032) 0:03:15.297 ******* 2019-12-09T09:07:28.022 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-09T09:07:28.079 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:28 +0000 (0:00:00.064) 0:03:15.362 ******* 2019-12-09T09:07:28.374 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2019-12-09T09:07:28.376 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.015125", "end": "2019-12-09 09:07:28.348300", "rc": 0, "start": "2019-12-09 09:07:28.333175", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-09T09:07:28.427 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:28 +0000 (0:00:00.345) 0:03:15.707 ******* 2019-12-09T09:07:28.439 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.016999", "end": "2019-12-09 09:07:28.393349", "rc": 0, "start": "2019-12-09 09:07:28.376350", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-09T09:07:28.497 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:28 +0000 (0:00:00.072) 0:03:15.780 ******* 2019-12-09T09:07:28.809 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2019-12-09T09:07:28.811 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.015577", "end": "2019-12-09 09:07:28.781348", "rc": 0, "start": "2019-12-09 09:07:28.765771", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:07:28.851 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:28 +0000 (0:00:00.353) 0:03:16.134 ******* 2019-12-09T09:07:28.858 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.014392", "end": "2019-12-09 09:07:28.810004", "rc": 0, "start": "2019-12-09 09:07:28.795612", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:07:28.914 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:28 +0000 (0:00:00.062) 0:03:16.197 ******* 2019-12-09T09:07:37.761 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2019-12-09T09:07:37.763 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:08.561523", "end": "2019-12-09 09:07:37.717347", "rc": 0, "start": "2019-12-09 09:07:29.155824", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:07:37.808 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:37 +0000 (0:00:08.893) 0:03:25.090 ******* 2019-12-09T09:07:37.975 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:08.690585", "end": "2019-12-09 09:07:37.941169", "rc": 0, "start": "2019-12-09 09:07:29.250584", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:07:38.015 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:38 +0000 (0:00:00.207) 0:03:25.298 ******* 2019-12-09T09:07:38.171 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2019-12-09T09:07:38.172 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-09T09:07:38.224 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:38 +0000 (0:00:00.208) 0:03:25.507 ******* 2019-12-09T09:07:38.264 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-09T09:07:38.266 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:07:38.312 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:38 +0000 (0:00:00.084) 0:03:25.591 ******* 2019-12-09T09:07:38.357 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-09T09:07:38.358 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:07:38.399 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:38 +0000 (0:00:00.090) 0:03:25.682 ******* 2019-12-09T09:07:38.407 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2019-12-09T09:07:38.407 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-09T09:07:38.460 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:38 +0000 (0:00:00.060) 0:03:25.743 ******* 2019-12-09T09:07:38.469 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-09T09:07:38.470 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-09T09:07:38.519 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:38 +0000 (0:00:00.057) 0:03:25.800 ******* 2019-12-09T09:07:38.527 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-09T09:07:38.528 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:07:38.569 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:38 +0000 (0:00:00.051) 0:03:25.852 ******* 2019-12-09T09:07:38.575 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-12-09T09:07:38.577 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:07:38.633 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:38 +0000 (0:00:00.063) 0:03:25.915 ******* 2019-12-09T09:07:38.642 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-09T09:07:38.643 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:07:38.688 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:38 +0000 (0:00:00.054) 0:03:25.970 ******* 2019-12-09T09:07:38.749 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-09T09:07:38.751 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-09T09:07:38.795 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:38 +0000 (0:00:00.107) 0:03:26.078 ******* 2019-12-09T09:07:38.834 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-12-09T09:07:38.836 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:07:38.889 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:38 +0000 (0:00:00.087) 0:03:26.165 ******* 2019-12-09T09:07:38.988 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-12-09T09:07:38.990 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:07:39.035 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:39 +0000 (0:00:00.152) 0:03:26.317 ******* 2019-12-09T09:07:39.191 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:07:39.232 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:39 +0000 (0:00:00.197) 0:03:26.514 ******* 2019-12-09T09:07:39.377 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-12-09T09:07:39.379 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-09T09:07:39.440 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:39 +0000 (0:00:00.207) 0:03:26.722 ******* 2019-12-09T09:07:39.468 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-12-09T09:07:39.470 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:07:39.507 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:39 +0000 (0:00:00.067) 0:03:26.790 ******* 2019-12-09T09:07:39.534 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-09T09:07:39.536 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:07:39.577 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:39 +0000 (0:00:00.069) 0:03:26.860 ******* 2019-12-09T09:07:39.590 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-09T09:07:39.641 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:39 +0000 (0:00:00.063) 0:03:26.923 ******* 2019-12-09T09:07:39.679 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:07:39.739 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:39 +0000 (0:00:00.097) 0:03:27.021 ******* 2019-12-09T09:07:39.768 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-09T09:07:39.769 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:07:39.810 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:39 +0000 (0:00:00.071) 0:03:27.093 ******* 2019-12-09T09:07:39.964 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2019-12-09T09:07:39.966 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-09T09:07:40.028 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:40 +0000 (0:00:00.217) 0:03:27.311 ******* 2019-12-09T09:07:40.194 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-09T09:07:40.238 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:40 +0000 (0:00:00.210) 0:03:27.521 ******* 2019-12-09T09:07:41.118 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-12-09T09:07:41.119 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-09T09:07:41.163 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:41 +0000 (0:00:00.924) 0:03:28.446 ******* 2019-12-09T09:07:41.205 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-12-09T09:07:41.206 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi022"}, "changed": false} 2019-12-09T09:07:41.246 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:41 +0000 (0:00:00.082) 0:03:28.528 ******* 2019-12-09T09:07:41.374 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-12-09T09:07:41.375 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-09T09:07:41.461 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:41 +0000 (0:00:00.215) 0:03:28.744 ******* 2019-12-09T09:07:41.516 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-12-09T09:07:41.516 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi035"}, "changed": false} 2019-12-09T09:07:41.583 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:41 +0000 (0:00:00.121) 0:03:28.866 ******* 2019-12-09T09:07:42.260 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2019-12-09T09:07:42.271 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi022.front.sepia.ceph.com", "ansible_hostname": "smithi022", "ansible_nodename": "smithi022"}, "changed": true, "name": "smithi022"} 2019-12-09T09:07:42.350 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:42 +0000 (0:00:00.766) 0:03:29.632 ******* 2019-12-09T09:07:42.360 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi035.front.sepia.ceph.com", "ansible_hostname": "smithi035", "ansible_nodename": "smithi035"}, "changed": true, "name": "smithi035"} 2019-12-09T09:07:42.418 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:42 +0000 (0:00:00.067) 0:03:29.700 ******* 2019-12-09T09:07:43.249 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-09T09:07:43.290 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:43 +0000 (0:00:00.872) 0:03:30.573 ******* 2019-12-09T09:07:43.292 INFO:teuthology.task.ansible.out: 2019-12-09T09:07:43.395 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-09T09:07:43.443 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:43 +0000 (0:00:00.152) 0:03:30.726 ******* 2019-12-09T09:07:44.232 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-12-09T09:07:44.234 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-09T09:07:44.276 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:44 +0000 (0:00:00.832) 0:03:31.559 ******* 2019-12-09T09:07:44.401 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-09T09:07:44.441 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:44 +0000 (0:00:00.164) 0:03:31.723 ******* 2019-12-09T09:07:45.078 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2019-12-09T09:07:45.080 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-09T09:07:45.120 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:45 +0000 (0:00:00.679) 0:03:32.403 ******* 2019-12-09T09:07:45.354 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-09T09:07:45.404 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:45 +0000 (0:00:00.283) 0:03:32.687 ******* 2019-12-09T09:07:46.147 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2019-12-09T09:07:46.148 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-09T09:07:46.194 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:46 +0000 (0:00:00.790) 0:03:33.477 ******* 2019-12-09T09:07:46.360 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-09T09:07:46.404 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:46 +0000 (0:00:00.209) 0:03:33.686 ******* 2019-12-09T09:07:46.538 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:07:46.595 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:46 +0000 (0:00:00.191) 0:03:33.878 ******* 2019-12-09T09:07:46.722 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:07:46.771 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:46 +0000 (0:00:00.174) 0:03:34.053 ******* 2019-12-09T09:07:46.937 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T09:07:46.939 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-09T09:07:46.984 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:46 +0000 (0:00:00.214) 0:03:34.267 ******* 2019-12-09T09:07:47.117 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-09T09:07:47.162 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:47 +0000 (0:00:00.176) 0:03:34.444 ******* 2019-12-09T09:07:47.340 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T09:07:47.341 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-09T09:07:47.386 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:47 +0000 (0:00:00.224) 0:03:34.669 ******* 2019-12-09T09:07:47.529 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-09T09:07:47.577 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:47 +0000 (0:00:00.190) 0:03:34.859 ******* 2019-12-09T09:07:48.363 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T09:07:48.364 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-09T09:07:48.402 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:48 +0000 (0:00:00.825) 0:03:35.685 ******* 2019-12-09T09:07:48.565 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-09T09:07:48.609 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:48 +0000 (0:00:00.206) 0:03:35.892 ******* 2019-12-09T09:07:49.412 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T09:07:49.413 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-09T09:07:49.452 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:49 +0000 (0:00:00.843) 0:03:36.735 ******* 2019-12-09T09:07:49.574 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-09T09:07:49.612 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:49 +0000 (0:00:00.159) 0:03:36.895 ******* 2019-12-09T09:07:51.886 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T09:07:51.886 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2019-12-09T09:07:51.888 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.099339", "end": "2019-12-09 09:07:51.849731", "rc": 0, "start": "2019-12-09 09:07:49.750392", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:07:51.931 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:51 +0000 (0:00:02.319) 0:03:39.214 ******* 2019-12-09T09:07:51.962 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-09T09:07:52.002 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:51 +0000 (0:00:00.070) 0:03:39.285 ******* 2019-12-09T09:07:52.138 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T09:07:52.140 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.191395", "end": "2019-12-09 09:07:52.097720", "rc": 0, "start": "2019-12-09 09:07:49.906325", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:07:52.201 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:52 +0000 (0:00:00.198) 0:03:39.484 ******* 2019-12-09T09:07:52.231 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-09T09:07:52.282 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:52 +0000 (0:00:00.081) 0:03:39.565 ******* 2019-12-09T09:07:53.013 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-09T09:07:53.015 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-09T09:07:53.018 INFO:teuthology.task.ansible.out: 2019-12-09T09:07:53.266 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-09T09:07:53.983 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-09T09:07:54.030 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:54 +0000 (0:00:01.748) 0:03:41.313 ******* 2019-12-09T09:07:54.059 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:07:54.096 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:54 +0000 (0:00:00.065) 0:03:41.378 ******* 2019-12-09T09:07:54.146 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-09T09:07:54.148 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:07:54.158 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:07:54.197 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:54 +0000 (0:00:00.101) 0:03:41.480 ******* 2019-12-09T09:07:54.204 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-09T09:07:54.205 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-09T09:07:54.252 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:54 +0000 (0:00:00.051) 0:03:41.531 ******* 2019-12-09T09:07:54.256 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:07:54.296 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:54 +0000 (0:00:00.047) 0:03:41.579 ******* 2019-12-09T09:07:54.303 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2019-12-09T09:07:54.306 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:07:54.351 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:54 +0000 (0:00:00.054) 0:03:41.633 ******* 2019-12-09T09:07:54.358 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:07:54.359 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-09T09:07:54.401 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:54 +0000 (0:00:00.050) 0:03:41.684 ******* 2019-12-09T09:07:54.409 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-09T09:07:54.411 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:07:54.421 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:07:54.475 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:54 +0000 (0:00:00.073) 0:03:41.757 ******* 2019-12-09T09:07:54.513 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-09T09:07:54.515 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:07:54.551 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:54 +0000 (0:00:00.076) 0:03:41.834 ******* 2019-12-09T09:07:54.588 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:07:54.590 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-09T09:07:54.626 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:54 +0000 (0:00:00.075) 0:03:41.909 ******* 2019-12-09T09:07:54.956 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2019-12-09T09:07:54.958 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-09T09:07:55.003 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:54 +0000 (0:00:00.374) 0:03:42.284 ******* 2019-12-09T09:07:55.135 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-09T09:07:55.198 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:55 +0000 (0:00:00.197) 0:03:42.481 ******* 2019-12-09T09:07:55.668 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2019-12-09T09:07:55.668 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.337066", "end": "2019-12-09 09:07:55.641502", "rc": 0, "start": "2019-12-09 09:07:55.304436", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: repos-va.psychz.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.math.princeton.edu\n * updates: mirror.cogentco.com\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: repos-va.psychz.net", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.math.princeton.edu", " * updates: mirror.cogentco.com", "No unfinished transactions left."]} 2019-12-09T09:07:55.712 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:55 +0000 (0:00:00.514) 0:03:42.995 ******* 2019-12-09T09:07:55.825 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.299226", "end": "2019-12-09 09:07:55.791047", "rc": 0, "start": "2019-12-09 09:07:55.491821", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: repos-va.psychz.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.math.princeton.edu\n * updates: mirror.cogentco.com\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: repos-va.psychz.net", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.math.princeton.edu", " * updates: mirror.cogentco.com", "No unfinished transactions left."]} 2019-12-09T09:07:55.863 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:55 +0000 (0:00:00.151) 0:03:43.146 ******* 2019-12-09T09:07:56.117 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-09T09:07:56.118 INFO:teuthology.task.ansible.out:fatal: [smithi022.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.058960", "end": "2019-12-09 09:07:56.087108", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:07:56.028148", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-12-09T09:07:56.160 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:56 +0000 (0:00:00.292) 0:03:43.439 ******* 2019-12-09T09:07:56.206 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2019-12-09T09:07:56.208 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:07:56.256 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:56 +0000 (0:00:00.097) 0:03:43.537 ******* 2019-12-09T09:07:56.260 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-09T09:07:56.262 INFO:teuthology.task.ansible.out:fatal: [smithi035.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.037671", "end": "2019-12-09 09:07:56.214587", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:07:56.176916", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} 2019-12-09T09:07:56.263 INFO:teuthology.task.ansible.out:...ignoring 2019-12-09T09:07:56.300 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:56 +0000 (0:00:00.045) 0:03:43.582 ******* 2019-12-09T09:07:56.338 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2019-12-09T09:07:56.340 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:07:56.389 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:07:56 +0000 (0:00:00.089) 0:03:43.672 ******* 2019-12-09T09:08:04.595 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-09T09:08:04.596 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-09T09:08:04.597 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-09T09:08:04.600 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:08:04.639 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:04 +0000 (0:00:08.249) 0:03:51.922 ******* 2019-12-09T09:08:05.271 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-09T09:08:05.280 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-09T09:08:05.282 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:08:05.316 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:05 +0000 (0:00:00.676) 0:03:52.598 ******* 2019-12-09T09:08:08.011 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T09:08:08.012 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-09T09:08:08.078 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:08 +0000 (0:00:02.759) 0:03:55.358 ******* 2019-12-09T09:08:08.925 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-09T09:08:08.960 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:08 +0000 (0:00:00.884) 0:03:56.243 ******* 2019-12-09T09:08:26.846 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:08:26.847 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-12-09T09:08:26.849 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-12-09T09:08:26.850 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: repos-va.psychz.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.math.princeton.edu\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Re 2019-12-09T09:08:26.852 INFO:teuthology.task.ansible.out:solved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 27 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-12-09T09:08:26.853 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7 2019-12-09T09:08:26.855 INFO:teuthology.task.ansible.out:.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:08:26.896 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:26 +0000 (0:00:17.936) 0:04:14.179 ******* 2019-12-09T09:08:27.721 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-12-09T09:08:27.722 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-09T09:08:27.762 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:27 +0000 (0:00:00.866) 0:04:15.045 ******* 2019-12-09T09:08:28.320 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-09T09:08:28.369 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:28 +0000 (0:00:00.606) 0:04:15.651 ******* 2019-12-09T09:08:28.419 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T09:08:28.420 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:28.458 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:28 +0000 (0:00:00.089) 0:04:15.741 ******* 2019-12-09T09:08:28.587 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:08:28.595 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-12-09T09:08:28.597 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-12-09T09:08:28.602 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: repos-va.psychz.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.math.princeton.edu\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Re 2019-12-09T09:08:28.603 INFO:teuthology.task.ansible.out:solved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 17 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-12-09T09:08:28.605 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:08:28.680 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:28 +0000 (0:00:00.221) 0:04:15.962 ******* 2019-12-09T09:08:28.939 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-12-09T09:08:28.943 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service dbus.service system.slice basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ip6tables.service iptables.service ipset.service ebtables.service shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-12-09T09:08:28.945 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-12-09T09:08:28.947 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "system.slice network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:08:29.014 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:29 +0000 (0:00:00.333) 0:04:16.296 ******* 2019-12-09T09:08:29.438 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ fatal: [smithi022.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-09T09:08:29.494 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:29 +0000 (0:00:00.479) 0:04:16.776 ******* 2019-12-09T09:08:29.526 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-12-09T09:08:29.527 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-09T09:08:29.569 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:29 +0000 (0:00:00.075) 0:04:16.852 ******* 2019-12-09T09:08:30.111 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-09T09:08:30.113 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-09T09:08:30.168 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:30 +0000 (0:00:00.598) 0:04:17.451 ******* 2019-12-09T09:08:30.218 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T09:08:30.220 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:30.274 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:30 +0000 (0:00:00.106) 0:04:17.557 ******* 2019-12-09T09:08:30.399 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-12-09T09:08:30.401 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-09T09:08:30.451 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:30 +0000 (0:00:00.176) 0:04:17.734 ******* 2019-12-09T09:08:30.493 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-12-09T09:08:30.495 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:30.552 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:30 +0000 (0:00:00.100) 0:04:17.834 ******* 2019-12-09T09:08:30.581 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-09T09:08:30.630 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:30 +0000 (0:00:00.078) 0:04:17.913 ******* 2019-12-09T09:08:30.659 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-12-09T09:08:30.660 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:30.702 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:30 +0000 (0:00:00.070) 0:04:17.983 ******* 2019-12-09T09:08:30.739 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-09T09:08:30.741 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:30.780 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:30 +0000 (0:00:00.075) 0:04:18.059 ******* 2019-12-09T09:08:30.786 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.service system.slice basic.target polkit.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "iptables.service ip6tables.service shutdown.target ebtables.service ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": " 2019-12-09T09:08:30.788 INFO:teuthology.task.ansible.out:yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPr 2019-12-09T09:08:30.797 INFO:teuthology.task.ansible.out:iority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:08:30.835 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:30 +0000 (0:00:00.059) 0:04:18.118 ******* 2019-12-09T09:08:30.841 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-12-09T09:08:30.843 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi022.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:30.881 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:30 +0000 (0:00:00.045) 0:04:18.164 ******* 2019-12-09T09:08:30.920 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:30.963 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:30 +0000 (0:00:00.081) 0:04:18.246 ******* 2019-12-09T09:08:30.992 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-09T09:08:31.030 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.064) 0:04:18.311 ******* 2019-12-09T09:08:31.061 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-12-09T09:08:31.063 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.110 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.082) 0:04:18.393 ******* 2019-12-09T09:08:31.140 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-12-09T09:08:31.142 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.187 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.076) 0:04:18.470 ******* 2019-12-09T09:08:31.214 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-12-09T09:08:31.216 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.261 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.073) 0:04:18.543 ******* 2019-12-09T09:08:31.267 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-12-09T09:08:31.268 INFO:teuthology.task.ansible.out:fatal: [smithi035.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-09T09:08:31.310 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.048) 0:04:18.592 ******* 2019-12-09T09:08:31.316 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T09:08:31.318 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.360 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.046) 0:04:18.639 ******* 2019-12-09T09:08:31.381 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-12-09T09:08:31.383 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.421 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.064) 0:04:18.703 ******* 2019-12-09T09:08:31.458 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:08:31.460 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.505 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.078) 0:04:18.781 ******* 2019-12-09T09:08:31.539 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-12-09T09:08:31.541 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.578 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.078) 0:04:18.860 ******* 2019-12-09T09:08:31.604 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-12-09T09:08:31.606 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.642 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.063) 0:04:18.924 ******* 2019-12-09T09:08:31.670 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.708 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.064) 0:04:18.989 ******* 2019-12-09T09:08:31.724 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-09T09:08:31.761 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.055) 0:04:19.044 ******* 2019-12-09T09:08:31.767 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-12-09T09:08:31.769 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.806 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.044) 0:04:19.089 ******* 2019-12-09T09:08:31.813 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-12-09T09:08:31.814 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.851 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.044) 0:04:19.134 ******* 2019-12-09T09:08:31.860 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-12-09T09:08:31.862 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:31.908 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.056) 0:04:19.190 ******* 2019-12-09T09:08:31.915 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-09T09:08:31.963 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:31 +0000 (0:00:00.055) 0:04:19.245 ******* 2019-12-09T09:08:31.973 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.012 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.048) 0:04:19.294 ******* 2019-12-09T09:08:32.019 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-12-09T09:08:32.021 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.063 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.051) 0:04:19.346 ******* 2019-12-09T09:08:32.069 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-12-09T09:08:32.071 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.120 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.057) 0:04:19.403 ******* 2019-12-09T09:08:32.140 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-09T09:08:32.144 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.187 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.066) 0:04:19.470 ******* 2019-12-09T09:08:32.194 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-09T09:08:32.195 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.240 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.053) 0:04:19.523 ******* 2019-12-09T09:08:32.248 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* skipping: [smithi035.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi035.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.289 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.048) 0:04:19.571 ******* 2019-12-09T09:08:32.295 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.344 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.053) 0:04:19.625 ******* 2019-12-09T09:08:32.357 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.402 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.059) 0:04:19.685 ******* 2019-12-09T09:08:32.408 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T09:08:32.412 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.452 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.049) 0:04:19.734 ******* 2019-12-09T09:08:32.460 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-09T09:08:32.500 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.048) 0:04:19.783 ******* 2019-12-09T09:08:32.507 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T09:08:32.508 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.558 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.057) 0:04:19.840 ******* 2019-12-09T09:08:32.564 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.603 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.045) 0:04:19.886 ******* 2019-12-09T09:08:32.609 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T09:08:32.610 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.655 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.052) 0:04:19.938 ******* 2019-12-09T09:08:32.662 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-12-09T09:08:32.664 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.706 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.050) 0:04:19.989 ******* 2019-12-09T09:08:32.712 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T09:08:32.714 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.768 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.062) 0:04:20.051 ******* 2019-12-09T09:08:32.780 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-12-09T09:08:32.781 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.818 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.049) 0:04:20.101 ******* 2019-12-09T09:08:32.824 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T09:08:32.826 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.863 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.045) 0:04:20.146 ******* 2019-12-09T09:08:32.869 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T09:08:32.871 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.916 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.050) 0:04:20.196 ******* 2019-12-09T09:08:32.921 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:32.960 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:32 +0000 (0:00:00.046) 0:04:20.242 ******* 2019-12-09T09:08:32.968 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-12-09T09:08:32.969 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.007 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.046) 0:04:20.289 ******* 2019-12-09T09:08:33.016 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T09:08:33.017 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.061 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.054) 0:04:20.344 ******* 2019-12-09T09:08:33.071 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:08:33.072 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.118 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.057) 0:04:20.401 ******* 2019-12-09T09:08:33.124 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-09T09:08:33.126 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.175 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.056) 0:04:20.457 ******* 2019-12-09T09:08:33.182 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-12-09T09:08:33.184 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.224 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.046) 0:04:20.504 ******* 2019-12-09T09:08:33.227 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:08:33.227 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.267 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.045) 0:04:20.549 ******* 2019-12-09T09:08:33.278 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-12-09T09:08:33.280 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.326 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.049) 0:04:20.599 ******* TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.364 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.048) 0:04:20.647 ******* 2019-12-09T09:08:33.370 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2019-12-09T09:08:33.372 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.420 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.055) 0:04:20.703 ******* 2019-12-09T09:08:33.426 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T09:08:33.427 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.470 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.050) 0:04:20.753 ******* 2019-12-09T09:08:33.477 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-12-09T09:08:33.477 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.525 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.054) 0:04:20.808 ******* 2019-12-09T09:08:33.556 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-12-09T09:08:33.557 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.613 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.087) 0:04:20.896 ******* 2019-12-09T09:08:33.651 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-12-09T09:08:33.653 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.700 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.087) 0:04:20.983 ******* 2019-12-09T09:08:33.745 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-12-09T09:08:33.746 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.789 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.088) 0:04:21.072 ******* 2019-12-09T09:08:33.796 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:08:33.801 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-09T09:08:33.852 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.062) 0:04:21.134 ******* 2019-12-09T09:08:33.858 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-09T09:08:33.860 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.899 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.047) 0:04:21.181 ******* 2019-12-09T09:08:33.936 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T09:08:33.938 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:33.976 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:33 +0000 (0:00:00.077) 0:04:21.259 ******* 2019-12-09T09:08:34.003 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T09:08:34.005 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.041 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.064) 0:04:21.324 ******* 2019-12-09T09:08:34.069 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T09:08:34.071 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.114 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.072) 0:04:21.397 ******* 2019-12-09T09:08:34.142 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T09:08:34.144 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.202 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.087) 0:04:21.484 ******* 2019-12-09T09:08:34.218 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:08:34.220 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:08:34.258 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.056) 0:04:21.541 ******* 2019-12-09T09:08:34.266 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T09:08:34.267 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.313 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.055) 0:04:21.596 ******* 2019-12-09T09:08:34.342 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T09:08:34.344 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.390 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.076) 0:04:21.672 ******* 2019-12-09T09:08:34.430 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-09T09:08:34.432 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.473 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.083) 0:04:21.756 ******* 2019-12-09T09:08:34.512 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.550 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.076) 0:04:21.832 ******* 2019-12-09T09:08:34.598 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-09T09:08:34.600 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.641 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.091) 0:04:21.924 ******* 2019-12-09T09:08:34.650 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:08:34.661 INFO:teuthology.task.ansible.out: [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-09T09:08:34.703 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.061) 0:04:21.985 ******* 2019-12-09T09:08:34.709 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:08:34.710 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.760 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.057) 0:04:22.043 ******* 2019-12-09T09:08:34.770 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T09:08:34.771 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.823 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.060) 0:04:22.104 ******* 2019-12-09T09:08:34.829 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.879 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.057) 0:04:22.161 ******* 2019-12-09T09:08:34.886 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T09:08:34.892 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.934 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.055) 0:04:22.217 ******* 2019-12-09T09:08:34.940 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T09:08:34.941 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:34.982 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:34 +0000 (0:00:00.047) 0:04:22.265 ******* 2019-12-09T09:08:35.004 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T09:08:35.005 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:35.066 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.083) 0:04:22.349 ******* 2019-12-09T09:08:35.096 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:08:35.097 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:35.144 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.076) 0:04:22.425 ******* 2019-12-09T09:08:35.181 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:08:35.183 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:35.235 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.092) 0:04:22.518 ******* 2019-12-09T09:08:35.277 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:35.314 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.078) 0:04:22.596 ******* 2019-12-09T09:08:35.341 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:35.379 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.064) 0:04:22.661 ******* 2019-12-09T09:08:35.385 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:08:35.386 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-09T09:08:35.430 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.051) 0:04:22.713 ******* 2019-12-09T09:08:35.442 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:08:35.443 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:35.517 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.086) 0:04:22.799 ******* 2019-12-09T09:08:35.549 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:08:35.550 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:35.616 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.094) 0:04:22.893 ******* 2019-12-09T09:08:35.658 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:35.697 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.086) 0:04:22.980 ******* 2019-12-09T09:08:35.727 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:35.764 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.066) 0:04:23.046 ******* 2019-12-09T09:08:35.801 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T09:08:35.803 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:35.859 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.095) 0:04:23.142 ******* 2019-12-09T09:08:35.870 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:08:35.872 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:08:35.919 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.059) 0:04:23.202 ******* 2019-12-09T09:08:35.925 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T09:08:35.926 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:35.995 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:35 +0000 (0:00:00.075) 0:04:23.278 ******* 2019-12-09T09:08:36.031 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-12-09T09:08:36.032 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.076 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.080) 0:04:23.358 ******* 2019-12-09T09:08:36.126 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-12-09T09:08:36.128 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.178 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.101) 0:04:23.460 ******* 2019-12-09T09:08:36.217 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-12-09T09:08:36.219 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.258 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.080) 0:04:23.541 ******* 2019-12-09T09:08:36.300 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-12-09T09:08:36.302 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.304 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-09T09:08:36.358 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.100) 0:04:23.641 ******* 2019-12-09T09:08:36.400 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.041) 0:04:23.682 ******* 2019-12-09T09:08:36.405 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-12-09T09:08:36.407 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.455 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.055) 0:04:23.738 ******* 2019-12-09T09:08:36.462 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T09:08:36.464 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.507 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.052) 0:04:23.790 ******* 2019-12-09T09:08:36.513 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-12-09T09:08:36.515 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.554 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.045) 0:04:23.835 ******* 2019-12-09T09:08:36.560 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T09:08:36.562 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.609 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.055) 0:04:23.891 ******* 2019-12-09T09:08:36.622 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.662 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.053) 0:04:23.945 ******* 2019-12-09T09:08:36.675 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.724 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.061) 0:04:24.006 ******* 2019-12-09T09:08:36.732 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-12-09T09:08:36.733 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.785 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.061) 0:04:24.068 ******* 2019-12-09T09:08:36.792 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:08:36.794 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.832 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.046) 0:04:24.114 ******* 2019-12-09T09:08:36.841 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.882 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.050) 0:04:24.165 ******* 2019-12-09T09:08:36.892 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:36.931 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.048) 0:04:24.213 ******* 2019-12-09T09:08:36.936 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:08:36.938 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-09T09:08:36.978 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:36 +0000 (0:00:00.047) 0:04:24.261 ******* 2019-12-09T09:08:36.986 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:08:36.988 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.034 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.056) 0:04:24.317 ******* 2019-12-09T09:08:37.041 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-09T09:08:37.042 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.043 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi022.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.055 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.057 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.098 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.064) 0:04:24.381 ******* 2019-12-09T09:08:37.104 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-09T09:08:37.107 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi022.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi022.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.152 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.053) 0:04:24.435 ******* 2019-12-09T09:08:37.158 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:08:37.160 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.203 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.050) 0:04:24.485 ******* 2019-12-09T09:08:37.244 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:08:37.246 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.300 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.097) 0:04:24.583 ******* 2019-12-09T09:08:37.328 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-12-09T09:08:37.329 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003915", "end": "2019-12-09 09:08:37.295234", "rc": 0, "start": "2019-12-09 09:08:37.291319", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-09T09:08:37.331 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.380 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.079) 0:04:24.662 ******* 2019-12-09T09:08:37.418 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.038) 0:04:24.700 ******* 2019-12-09T09:08:37.434 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-12-09T09:08:37.436 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.486 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.067) 0:04:24.768 ******* 2019-12-09T09:08:37.500 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.562 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.074) 0:04:24.843 ******* 2019-12-09T09:08:37.568 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-12-09T09:08:37.572 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.630 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.068) 0:04:24.913 ******* 2019-12-09T09:08:37.640 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T09:08:37.641 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.684 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.054) 0:04:24.967 ******* 2019-12-09T09:08:37.690 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-12-09T09:08:37.692 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.736 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.051) 0:04:25.019 ******* 2019-12-09T09:08:37.743 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T09:08:37.745 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.785 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.049) 0:04:25.068 ******* 2019-12-09T09:08:37.795 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.844 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.058) 0:04:25.127 ******* 2019-12-09T09:08:37.850 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-12-09T09:08:37.852 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:37.901 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.055) 0:04:25.182 ******* 2019-12-09T09:08:37.944 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.004 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:37 +0000 (0:00:00.103) 0:04:25.286 ******* 2019-12-09T09:08:38.042 INFO:teuthology.task.ansible.out: 2019-12-09T09:08:38.044 INFO:teuthology.task.ansible.out:TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.045 INFO:teuthology.task.ansible.out: 2019-12-09T09:08:38.106 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.102) 0:04:25.389 ******* 2019-12-09T09:08:38.134 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.172 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.065) 0:04:25.455 ******* 2019-12-09T09:08:38.199 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-12-09T09:08:38.201 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.247 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.074) 0:04:25.530 ******* 2019-12-09T09:08:38.274 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-12-09T09:08:38.276 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.330 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.082) 0:04:25.612 ******* 2019-12-09T09:08:38.357 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-12-09T09:08:38.359 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-09T09:08:38.424 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.090) 0:04:25.703 ******* 2019-12-09T09:08:38.427 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.469 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.049) 0:04:25.752 ******* 2019-12-09T09:08:38.476 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T09:08:38.477 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item={u'block_used': 13040464, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284020, u'size_available': 930955345920, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966947, u'device': u'/dev/sda1', u'inode_used': 88029, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284020, "block_size": 4096, "block_total": 240324484, "block_used": 13040464, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966947, "inode_total": 61054976, "inode_used": 88029, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955345920, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-09T09:08:38.525 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.049) 0:04:25.802 ******* 2019-12-09T09:08:38.527 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.564 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.045) 0:04:25.847 ******* 2019-12-09T09:08:38.590 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-12-09T09:08:38.592 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.628 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.063) 0:04:25.911 ******* 2019-12-09T09:08:38.659 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:08:38.661 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-09T09:08:38.712 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.076) 0:04:25.988 ******* 2019-12-09T09:08:38.764 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-09T09:08:38.765 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.777 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-12-09T09:08:38.789 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.800 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.813 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.824 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.825 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-09T09:08:38.898 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.192) 0:04:26.180 ******* 2019-12-09T09:08:38.903 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-09T09:08:38.905 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011830", "end": "2019-12-09 09:08:38.835394", "rc": 0, "start": "2019-12-09 09:08:38.823564", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:38.944 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:38 +0000 (0:00:00.046) 0:04:26.227 ******* 2019-12-09T09:08:39.002 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-12-09T09:08:39.004 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'block_used': 13040464, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284020, u'size_available': 930955345920, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966947, u'device': u'/dev/sda1', u'inode_used': 88029, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284020, "block_size": 4096, "block_total": 240324484, "block_used": 13040464, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966947, "inode_total": 61054976, "inode_used": 88029, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955345920, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:39.048 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:39 +0000 (0:00:00.099) 0:04:26.326 ******* 2019-12-09T09:08:39.063 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-12-09T09:08:39.068 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004789", "end": "2019-12-09 09:08:39.035385", "rc": 0, "start": "2019-12-09 09:08:39.030596", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-09T09:08:39.124 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:39 +0000 (0:00:00.076) 0:04:26.403 ******* 2019-12-09T09:08:39.126 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-09T09:08:39.178 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:39 +0000 (0:00:00.057) 0:04:26.460 ******* 2019-12-09T09:08:39.183 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-12-09T09:08:39.184 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:39.242 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:39 +0000 (0:00:00.064) 0:04:26.525 ******* 2019-12-09T09:08:39.252 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:08:39.256 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQJ7', u'wwn-0x5000c500871a6c97'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4HQJ7', u'wwn': u'0x5000c500871a6c97', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQJ7-part1', u'wwn-0x5000c500871a6c97-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQJ7", "wwn-0x5000c500871a6c97"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQJ7-part1", "wwn-0x5000c500871a6c97-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HQJ7", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T09:08:39.257 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500871a6c97"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:39.289 INFO:teuthology.task.ansible.out: 2019-12-09T09:08:39.291 INFO:teuthology.task.ansible.out:TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:39.335 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:39 +0000 (0:00:00.092) 0:04:26.618 ******* 2019-12-09T09:08:39.337 INFO:teuthology.task.ansible.out: 2019-12-09T09:08:39.386 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-12-09T09:08:39.387 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:39.460 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:39 +0000 (0:00:00.113) 0:04:26.732 ******* 2019-12-09T09:08:39.485 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:39.518 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:39 +0000 (0:00:00.068) 0:04:26.800 ******* 2019-12-09T09:08:40.028 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-12-09T09:08:40.028 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-09T09:08:40.069 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:40 +0000 (0:00:00.551) 0:04:27.351 ******* 2019-12-09T09:08:40.118 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T09:08:40.119 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item={u'block_used': 13040464, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284020, u'size_available': 930955345920, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284020, "block_size": 4096, "block_total": 240324484, "block_used": 13040464, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955345920, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-09T09:08:40.162 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:40 +0000 (0:00:00.093) 0:04:27.445 ******* 2019-12-09T09:08:40.506 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-09T09:08:40.507 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.013765", "end": "2019-12-09 09:08:40.476443", "rc": 0, "start": "2019-12-09 09:08:40.462678", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:40.548 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:40 +0000 (0:00:00.382) 0:04:27.827 ******* 2019-12-09T09:08:40.602 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-12-09T09:08:40.605 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'block_used': 13040464, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284020, u'size_available': 930955345920, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284020, "block_size": 4096, "block_total": 240324484, "block_used": 13040464, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955345920, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:40.651 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:40 +0000 (0:00:00.106) 0:04:27.934 ******* 2019-12-09T09:08:40.690 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-09T09:08:40.729 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:40 +0000 (0:00:00.077) 0:04:28.012 ******* 2019-12-09T09:08:40.782 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:08:40.790 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J2TL', u'wwn-0x5000c500871b6b83'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4J2TL', u'wwn': u'0x5000c500871b6b83', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J2TL-part1', u'wwn-0x5000c500871b6b83-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2TL", "wwn-0x5000c500871b6b83"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2TL-part1", "wwn-0x5000c500871b6b83-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J2TL", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T09:08:40.792 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500871b6b83"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:41.036 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-y9T4jZ-Z2MG-TyCL-7h1x-10Cs-id1t-sL0a66', u'nvme-INTEL_SSDPEDMD400G4_CVFT53300063400BGN', u'nvme-nvme.8086-43564654353333303030363334303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.464845", "end": "2019-12-09 09:08:40.994262", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-y9T4jZ-Z2MG-TyCL-7h1x-10Cs-id1t-sL0a66", "nvme-INTEL_SSDPEDMD400G4_CVFT53300063400BGN", "nvme-nvme.8086-43564654353333303030363334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:39.529 2019-12-09T09:08:41.037 INFO:teuthology.task.ansible.out:417", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:08:42.407 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebXbyYw1m2GXqfNZ4dP2SveyjMI5eYnpyy'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.053885", "end": "2019-12-09 09:08:42.354850", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebXbyYw1m2GXqfNZ4dP2SveyjMI5eYnpyy"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:41.300965", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:08:42.599 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-zhB33L-aan7-CFDC-a8GQ-9D60-Wfz7-9arZFh', u'nvme-INTEL_SSDPEDMX012T4_CVPD5170000L1P2LGN', u'nvme-nvme.8086-43565044353137303030304c3150324c474e-494e54454c205353445045444d583031325434-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'1.09 TB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'2344225968', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMX012T4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.487061", "end": "2019-12-09 09:08:42.562016", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-zhB33L-aan7-CFDC-a8GQ-9D60-Wfz7-9arZFh", "nvme-INTEL_SSDPEDMX012T4_CVPD5170000L1P2LGN", "nvme-nvme.8086-43565044353137303030304c3150324c474e-494e54454c205353445045444d583031325434-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMX012T4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "2344225968", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "1.09 TB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:41.07495 2019-12-09T09:08:42.601 INFO:teuthology.task.ansible.out:5", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:08:43.792 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHeb0EaWzcPpFFRsiUZZFqRE9l2DALF3V1Cc'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.068440", "end": "2019-12-09 09:08:43.754713", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHeb0EaWzcPpFFRsiUZZFqRE9l2DALF3V1Cc"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:42.686273", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:08:44.035 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Z7kahzeFVOmMgXJ6y5h78jQGp4rUYMq89'], u'uuids': [u'122c49eb-695e-41c3-93b7-f44b504168ee']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.064993", "end": "2019-12-09 09:08:43.999366", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Z7kahzeFVOmMgXJ6y5h78jQGp4rUYMq89"], "labels": [], "masters": [], "uuids": ["122c49eb-695e-41c3-93b7-f44b504168ee"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:42.934373", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:08:45.143 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHeb8wdjhOum1UF3VVJKAQzOPq6AOIdo5giQ'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.046178", "end": "2019-12-09 09:08:45.112940", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHeb8wdjhOum1UF3VVJKAQzOPq6AOIdo5giQ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:44.066762", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:08:45.443 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Z1cLuswwmMdtKlNHOd5kZfCLJkY5exq9E'], u'uuids': [u'3a1e852d-60fb-4b3f-85ce-57e67922699d']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.054177", "end": "2019-12-09 09:08:45.407622", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Z1cLuswwmMdtKlNHOd5kZfCLJkY5exq9E"], "labels": [], "masters": [], "uuids": ["3a1e852d-60fb-4b3f-85ce-57e67922699d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:44.353445", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:08:46.519 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebb38Se2Rfcu02LCwtbtuJExfuuEwoFxp0'], u'uuids': [u'67fbfc30-38d8-482a-9f00-d3fd17a2397b']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.055797", "end": "2019-12-09 09:08:46.480922", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebb38Se2Rfcu02LCwtbtuJExfuuEwoFxp0"], "labels": [], "masters": [], "uuids": ["67fbfc30-38d8-482a-9f00-d3fd17a2397b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:45.425125", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:08:46.869 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8ZPR9ZubfsktUSzXd9edazVhiAQ5oigCbg'], u'uuids': [u'bef9486a-5f5f-4c36-8fa2-f47b8700bb52']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.071926", "end": "2019-12-09 09:08:46.833385", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8ZPR9ZubfsktUSzXd9edazVhiAQ5oigCbg"], "labels": [], "masters": [], "uuids": ["bef9486a-5f5f-4c36-8fa2-f47b8700bb52"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:45.761459", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:08:47.897 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebQl03sayjR5z2EdPGQDjTGhEuYUhDYydH'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.052410", "end": "2019-12-09 09:08:47.863741", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebQl03sayjR5z2EdPGQDjTGhEuYUhDYydH"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:46.811331", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:08:47.981 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:47 +0000 (0:00:07.251) 0:04:35.263 ******* 2019-12-09T09:08:48.041 INFO:teuthology.task.ansible.out: 2019-12-09T09:08:48.043 INFO:teuthology.task.ansible.out:TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:08:48.044 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQJ7', u'wwn-0x5000c500871a6c97'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4HQJ7', u'wwn': u'0x5000c500871a6c97', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQJ7-part1', u'wwn-0x5000c500871a6c97-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQJ7", "wwn-0x5000c500871a6c97"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQJ7-part1", "wwn-0x5000c500871a6c97-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HQJ7", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T09:08:48.046 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500871a6c97"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:48.260 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8ZbW4F8XbW2TjpXoTHRvkXaD78qBw5BhAt'], u'uuids': [u'b2010dd9-c7d8-4432-bb7c-ecd67057fe72']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.063589", "end": "2019-12-09 09:08:48.225109", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8ZbW4F8XbW2TjpXoTHRvkXaD78qBw5BhAt"], "labels": [], "masters": [], "uuids": ["b2010dd9-c7d8-4432-bb7c-ecd67057fe72"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:47.161520", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition th 2019-12-09T09:08:48.262 INFO:teuthology.task.ansible.out:e disk using fdisk or", "other utilities."]} 2019-12-09T09:08:48.450 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:08:48.451 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-y9T4jZ-Z2MG-TyCL-7h1x-10Cs-id1t-sL0a66', u'nvme-INTEL_SSDPEDMD400G4_CVFT53300063400BGN', u'nvme-nvme.8086-43564654353333303030363334303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.087240", "end": "2019-12-09 09:08:48.411545", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-y9T4jZ-Z2MG-TyCL-7h1x-10Cs-id1t-sL0a66", "nvme-INTEL_SSDPEDMD400G4_CVFT53300063400BGN", "nvme-nvme.8086-43564654353333303030363334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:48.324305", "stderr": 2019-12-09T09:08:48.452 INFO:teuthology.task.ansible.out: "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0806234 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0806234 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:48.920 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebXbyYw1m2GXqfNZ4dP2SveyjMI5eYnpyy'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.167997", "end": "2019-12-09 09:08:48.880994", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebXbyYw1m2GXqfNZ4dP2SveyjMI5eYnpyy"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:48.712997", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.162832 s, 708 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.162832 s, 708 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:49.408 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHeb0EaWzcPpFFRsiUZZFqRE9l2DALF3V1Cc'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.178700", "end": "2019-12-09 09:08:49.377439", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHeb0EaWzcPpFFRsiUZZFqRE9l2DALF3V1Cc"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:49.198739", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.172213 s, 670 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.172213 s, 670 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:49.634 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Zgye8rTtPxM7rTwiefrfoP2xxBDidvgEF'], u'uuids': [u'5bdd116c-a679-4baa-ba77-ee68e8e7e910']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.063776", "end": "2019-12-09 09:08:49.591993", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Zgye8rTtPxM7rTwiefrfoP2xxBDidvgEF"], "labels": [], "masters" 2019-12-09T09:08:49.636 INFO:teuthology.task.ansible.out:: [], "uuids": ["5bdd116c-a679-4baa-ba77-ee68e8e7e910"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:48.528217", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:08:49.722 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:49 +0000 (0:00:01.738) 0:04:37.002 ******* 2019-12-09T09:08:49.772 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J2TL', u'wwn-0x5000c500871b6b83'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4J2TL', u'wwn': u'0x5000c500871b6b83', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J2TL-part1', u'wwn-0x5000c500871b6b83-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2TL", "wwn-0x5000c500871b6b83"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2TL-part1", "wwn-0x5000c500871b6b83-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": " 2019-12-09T09:08:49.774 INFO:teuthology.task.ansible.out:512", "serial": "Z1W4J2TL", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b6b83"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:49.908 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHeb8wdjhOum1UF3VVJKAQzOPq6AOIdo5giQ'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.174728", "end": "2019-12-09 09:08:49.881233", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHeb8wdjhOum1UF3VVJKAQzOPq6AOIdo5giQ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:49.706505", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.168459 s, 685 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.168459 s, 685 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:50.182 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-zhB33L-aan7-CFDC-a8GQ-9D60-Wfz7-9arZFh', u'nvme-INTEL_SSDPEDMX012T4_CVPD5170000L1P2LGN', u'nvme-nvme.8086-43565044353137303030304c3150324c474e-494e54454c205353445045444d583031325434-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'1.09 TB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'2344225968', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMX012T4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.087903", "end": "2019-12-09 09:08:50.150662", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-zhB33L-aan7-CFDC-a8GQ-9D60-Wfz7-9arZFh", "nvme-INTEL_SSDPEDMX012T4_CVPD5170000L1P2LGN", "nvme-nvme.8086-43565044353137303030304c3150324c474e-494e54454c205353445045444d583031325434-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMX012T4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "2344225968", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "1.09 TB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:50.062759", "stderr": " 2019-12-09T09:08:50.184 INFO:teuthology.task.ansible.out:110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0810041 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0810041 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:50.411 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebb38Se2Rfcu02LCwtbtuJExfuuEwoFxp0'], u'uuids': [u'67fbfc30-38d8-482a-9f00-d3fd17a2397b']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.168631", "end": "2019-12-09 09:08:50.378470", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebb38Se2Rfcu02LCwtbtuJExfuuEwoFxp0"], "labels": [], "masters": [], "uuids": ["67fbfc30-38d8-482a-9f00-d3fd17a2397b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:50.209839", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.162303 s, 711 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.162303 s, 711 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:50.665 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Z7kahzeFVOmMgXJ6y5h78jQGp4rUYMq89'], u'uuids': [u'122c49eb-695e-41c3-93b7-f44b504168ee']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.165717", "end": "2019-12-09 09:08:50.626554", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Z7kahzeFVOmMgXJ6y5h78jQGp4rUYMq89"], "labels": [], "masters": [], "uuids": ["122c49eb-695e-41c3-93b7-f44b504168ee"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:50.460837", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.159505 s, 723 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.159505 s, 723 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:50.910 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebQl03sayjR5z2EdPGQDjTGhEuYUhDYydH'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT53300063400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.189946", "end": "2019-12-09 09:08:50.870810", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-SuV3f4s33kiK0NzzHHpKtZ0uUCk6MHebQl03sayjR5z2EdPGQDjTGhEuYUhDYydH"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT53300063400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:50.680864", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.183173 s, 630 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.183173 s, 630 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:50.963 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:50 +0000 (0:00:01.243) 0:04:38.246 ******* 2019-12-09T09:08:51.143 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Z1cLuswwmMdtKlNHOd5kZfCLJkY5exq9E'], u'uuids': [u'3a1e852d-60fb-4b3f-85ce-57e67922699d']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.182148", "end": "2019-12-09 09:08:51.102542", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Z1cLuswwmMdtKlNHOd5kZfCLJkY5exq9E"], "labels": [], "masters": [], "uuids": ["3a1e852d-60fb-4b3f-85ce-57e67922699d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:50.920394", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.175871 s, 656 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.175871 s, 656 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:51.375 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2019-12-09T09:08:51.377 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.084938", "end": "2019-12-09 09:08:51.341962", "rc": 0, "start": "2019-12-09 09:08:51.257024", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:51.428 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:51 +0000 (0:00:00.460) 0:04:38.707 ******* 2019-12-09T09:08:51.666 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:08:51.667 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8ZPR9ZubfsktUSzXd9edazVhiAQ5oigCbg'], u'uuids': [u'bef9486a-5f5f-4c36-8fa2-f47b8700bb52']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.195370", "end": "2019-12-09 09:08:51.618450", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8ZPR9ZubfsktUSzXd9edazVhiAQ5oigCbg"], "labels": [], "masters": [], "uuids": ["bef9486a-5f5f-4c36-8fa2-f47b8700bb52"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:51.423080", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.188862 s, 611 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.188862 s, 611 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:51.848 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-12-09T09:08:51.850 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.062746", "end": "2019-12-09 09:08:51.812031", "rc": 0, "start": "2019-12-09 09:08:51.749285", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:51.886 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:51 +0000 (0:00:00.462) 0:04:39.169 ******* 2019-12-09T09:08:51.916 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-09T09:08:51.987 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:51 +0000 (0:00:00.100) 0:04:39.269 ******* 2019-12-09T09:08:52.029 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:52.069 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:52 +0000 (0:00:00.081) 0:04:39.351 ******* 2019-12-09T09:08:52.108 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:52.167 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:52 +0000 (0:00:00.098) 0:04:39.450 ******* 2019-12-09T09:08:52.177 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:08:52.179 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8ZbW4F8XbW2TjpXoTHRvkXaD78qBw5BhAt'], u'uuids': [u'b2010dd9-c7d8-4432-bb7c-ecd67057fe72']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.171741", "end": "2019-12-09 09:08:52.079296", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8ZbW4F8XbW2TjpXoTHRvkXaD78qBw5BhAt"], "labels": [], "masters": [], "uuids": ["b2010dd9-c7d8-4432-bb7c-ecd67057fe72"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:51.907555", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.165374 s, 697 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.165374 s, 697 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:52.190 INFO:teuthology.task.ansible.out: 2019-12-09T09:08:52.192 INFO:teuthology.task.ansible.out:TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:52.232 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:52 +0000 (0:00:00.065) 0:04:39.515 ******* 2019-12-09T09:08:52.276 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-12-09T09:08:52.278 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:52.327 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:52 +0000 (0:00:00.093) 0:04:39.609 ******* 2019-12-09T09:08:52.631 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:08:52.632 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Zgye8rTtPxM7rTwiefrfoP2xxBDidvgEF'], u'uuids': [u'5bdd116c-a679-4baa-ba77-ee68e8e7e910']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVPD5170000L1P2LGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.197880", "end": "2019-12-09 09:08:52.588845", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-XPMiPiIgSlxwPrUFmIiVqvj7SPfTZd8Zgye8rTtPxM7rTwiefrfoP2xxBDidvgEF"], "labels": [], "masters": [], "uuids": ["5bdd116c-a679-4baa-ba77-ee68e8e7e910"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVPD5170000L1P2LGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:08:52.390965", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.191207 s, 603 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.191207 s, 603 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:52.694 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:52 +0000 (0:00:00.363) 0:04:39.973 ******* 2019-12-09T09:08:52.697 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi022.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:08:52.755 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:52 +0000 (0:00:00.065) 0:04:40.038 ******* 2019-12-09T09:08:52.807 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T09:08:52.808 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={u'block_used': 13040464, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284020, u'size_available': 930955345920, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966947, u'device': u'/dev/sda1', u'inode_used': 88029, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284020, "block_size": 4096, "block_total": 240324484, "block_used": 13040464, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966947, "inode_total": 61054976, "inode_used": 88029, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955345920, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:52.874 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:52 +0000 (0:00:00.118) 0:04:40.156 ******* 2019-12-09T09:08:52.914 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:52.957 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:52 +0000 (0:00:00.080) 0:04:40.237 ******* 2019-12-09T09:08:52.993 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:53.036 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.078) 0:04:40.316 ******* 2019-12-09T09:08:53.071 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:08:53.073 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:53.121 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.087) 0:04:40.403 ******* 2019-12-09T09:08:53.131 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi035.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.081200", "end": "2019-12-09 09:08:53.061537", "rc": 0, "start": "2019-12-09 09:08:52.980337", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:53.173 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.051) 0:04:40.455 ******* 2019-12-09T09:08:53.179 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-12-09T09:08:53.180 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:53.225 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.052) 0:04:40.508 ******* 2019-12-09T09:08:53.265 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-09T09:08:53.267 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:53.302 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.077) 0:04:40.585 ******* 2019-12-09T09:08:53.340 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-09T09:08:53.342 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:53.384 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.076) 0:04:40.662 ******* 2019-12-09T09:08:53.580 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-12-09T09:08:53.580 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.057433", "end": "2019-12-09 09:08:53.548044", "rc": 0, "start": "2019-12-09 09:08:53.490611", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:08:53.623 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.243) 0:04:40.905 ******* 2019-12-09T09:08:53.652 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-09T09:08:53.696 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.073) 0:04:40.979 ******* 2019-12-09T09:08:53.735 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-12-09T09:08:53.736 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:53.775 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.079) 0:04:41.058 ******* 2019-12-09T09:08:53.802 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-12-09T09:08:53.803 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:53.841 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.065) 0:04:41.124 ******* 2019-12-09T09:08:53.868 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T09:08:53.869 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:53.909 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.067) 0:04:41.192 ******* 2019-12-09T09:08:53.949 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:53.992 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:53 +0000 (0:00:00.078) 0:04:41.270 ******* 2019-12-09T09:08:54.252 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-12-09T09:08:54.253 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-09T09:08:54.305 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:54 +0000 (0:00:00.317) 0:04:41.587 ******* 2019-12-09T09:08:54.322 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-12-09T09:08:54.323 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:08:54.363 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:54 +0000 (0:00:00.057) 0:04:41.645 ******* 2019-12-09T09:08:54.410 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T09:08:54.412 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={u'block_used': 13040464, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284020, u'size_available': 930955345920, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284020, "block_size": 4096, "block_total": 240324484, "block_used": 13040464, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955345920, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:54.457 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:54 +0000 (0:00:00.094) 0:04:41.740 ******* 2019-12-09T09:08:54.513 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:54.564 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:54 +0000 (0:00:00.107) 0:04:41.847 ******* 2019-12-09T09:08:54.593 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:54.653 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:54 +0000 (0:00:00.088) 0:04:41.936 ******* 2019-12-09T09:08:54.685 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:54.726 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:54 +0000 (0:00:00.072) 0:04:42.009 ******* 2019-12-09T09:08:54.769 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-12-09T09:08:54.771 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:54.826 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:54 +0000 (0:00:00.099) 0:04:42.108 ******* 2019-12-09T09:08:54.863 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-09T09:08:54.865 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:54.906 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:54 +0000 (0:00:00.080) 0:04:42.189 ******* 2019-12-09T09:08:54.955 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-09T09:08:54.956 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:54.996 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:54 +0000 (0:00:00.090) 0:04:42.279 ******* 2019-12-09T09:08:55.136 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:08:55.540 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-12-09T09:08:55.542 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-09T09:08:55.582 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:55 +0000 (0:00:00.585) 0:04:42.864 ******* 2019-12-09T09:08:55.657 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:08:55.659 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:08:56.132 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:08:56.167 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:08:56.620 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:08:56.654 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:08:57.099 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:08:57.133 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:08:57.186 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:57 +0000 (0:00:01.604) 0:04:44.469 ******* 2019-12-09T09:08:57.506 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T09:08:57.508 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-09T09:08:57.544 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:57 +0000 (0:00:00.357) 0:04:44.827 ******* 2019-12-09T09:08:57.594 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:08:57.595 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:57.620 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:08:57.884 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:08:57.886 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:08:58.084 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:08:58.089 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:08:58.126 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:58 +0000 (0:00:00.582) 0:04:45.409 ******* 2019-12-09T09:08:58.209 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:08:58.450 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-09T09:08:58.487 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:58 +0000 (0:00:00.361) 0:04:45.770 ******* 2019-12-09T09:08:58.529 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:08:58.531 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:08:58.543 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:08:58.872 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:08:58.874 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:08:58.930 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:58 +0000 (0:00:00.442) 0:04:46.212 ******* 2019-12-09T09:08:59.185 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:08:59.279 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-12-09T09:08:59.281 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:08:59.319 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:59 +0000 (0:00:00.389) 0:04:46.602 ******* 2019-12-09T09:08:59.357 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-12-09T09:08:59.359 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-09T09:08:59.410 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:59 +0000 (0:00:00.090) 0:04:46.692 ******* 2019-12-09T09:08:59.507 INFO:teuthology.task.ansible.out: 2019-12-09T09:08:59.508 INFO:teuthology.task.ansible.out:TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:08:59.510 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:08:59.815 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:08:59.871 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:08:59 +0000 (0:00:00.460) 0:04:47.153 ******* 2019-12-09T09:09:00.166 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-12-09T09:09:00.167 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:09:00.220 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:00 +0000 (0:00:00.347) 0:04:47.501 ******* 2019-12-09T09:09:00.225 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-12-09T09:09:00.227 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true} 2019-12-09T09:09:00.275 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:00 +0000 (0:00:00.056) 0:04:47.557 ******* 2019-12-09T09:09:00.281 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-12-09T09:09:00.289 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-09T09:09:00.327 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:00 +0000 (0:00:00.052) 0:04:47.610 ******* 2019-12-09T09:09:00.771 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-12-09T09:09:00.772 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true} 2019-12-09T09:09:00.812 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:00 +0000 (0:00:00.485) 0:04:48.095 ******* 2019-12-09T09:09:01.038 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-12-09T09:09:01.040 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-09T09:09:01.077 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.264) 0:04:48.359 ******* 2019-12-09T09:09:01.106 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2019-12-09T09:09:01.108 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.145 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.067) 0:04:48.427 ******* 2019-12-09T09:09:01.173 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-12-09T09:09:01.175 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.219 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.073) 0:04:48.501 ******* 2019-12-09T09:09:01.257 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-12-09T09:09:01.259 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.313 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.094) 0:04:48.596 ******* 2019-12-09T09:09:01.323 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-12-09T09:09:01.324 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-09T09:09:01.367 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.053) 0:04:48.650 ******* 2019-12-09T09:09:01.374 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-12-09T09:09:01.376 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.415 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.047) 0:04:48.697 ******* 2019-12-09T09:09:01.422 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2019-12-09T09:09:01.424 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.471 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.056) 0:04:48.754 ******* 2019-12-09T09:09:01.480 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2019-12-09T09:09:01.482 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.520 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.048) 0:04:48.803 ******* 2019-12-09T09:09:01.527 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-12-09T09:09:01.529 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.577 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.050) 0:04:48.853 ******* 2019-12-09T09:09:01.579 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.634 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.063) 0:04:48.917 ******* 2019-12-09T09:09:01.647 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.698 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.062) 0:04:48.979 ******* 2019-12-09T09:09:01.703 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-12-09T09:09:01.705 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.757 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.060) 0:04:49.040 ******* 2019-12-09T09:09:01.768 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.808 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.051) 0:04:49.091 ******* 2019-12-09T09:09:01.835 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2019-12-09T09:09:01.837 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.885 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.076) 0:04:49.168 ******* 2019-12-09T09:09:01.913 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2019-12-09T09:09:01.914 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:01.961 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:01 +0000 (0:00:00.071) 0:04:49.240 ******* 2019-12-09T09:09:01.998 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:02.049 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:02 +0000 (0:00:00.091) 0:04:49.331 ******* 2019-12-09T09:09:02.278 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-12-09T09:09:02.279 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-09T09:09:02.329 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:02 +0000 (0:00:00.280) 0:04:49.612 ******* 2019-12-09T09:09:02.379 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2019-12-09T09:09:02.387 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:02.421 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:02 +0000 (0:00:00.092) 0:04:49.704 ******* 2019-12-09T09:09:02.451 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2019-12-09T09:09:02.453 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:02.494 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:02 +0000 (0:00:00.072) 0:04:49.777 ******* 2019-12-09T09:09:02.567 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-12-09T09:09:02.568 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-09T09:09:02.641 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:02 +0000 (0:00:00.146) 0:04:49.923 ******* 2019-12-09T09:09:02.667 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2019-12-09T09:09:02.669 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:02.713 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:02 +0000 (0:00:00.072) 0:04:49.995 ******* 2019-12-09T09:09:02.754 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:02.793 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:02 +0000 (0:00:00.079) 0:04:50.075 ******* 2019-12-09T09:09:03.512 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-12-09T09:09:03.512 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-09T09:09:03.588 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:03 +0000 (0:00:00.795) 0:04:50.871 ******* 2019-12-09T09:09:03.590 INFO:teuthology.task.ansible.out: 2019-12-09T09:09:03.619 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-12-09T09:09:03.620 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:03.671 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:03 +0000 (0:00:00.082) 0:04:50.954 ******* 2019-12-09T09:09:03.786 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-12-09T09:09:03.787 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-09T09:09:03.838 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:03 +0000 (0:00:00.166) 0:04:51.120 ******* 2019-12-09T09:09:03.876 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-12-09T09:09:03.878 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:03.942 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:03 +0000 (0:00:00.104) 0:04:51.225 ******* 2019-12-09T09:09:04.241 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-12-09T09:09:04.244 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target system.slice sntp.service syslog.target tmp.mount systemd-journald.socket -.mount ntpdate.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "multi-user.target chronyd.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-12-09T09:09:04.246 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "-.mount basic.target", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-12-09T09:09:04.247 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:09:04.294 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:04 +0000 (0:00:00.351) 0:04:51.577 ******* 2019-12-09T09:09:04.446 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "systemd-journald.socket ntpdate.service sntp.service tmp.mount basic.target syslog.target system.slice -.mount", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target chronyd.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-12-09T09:09:04.448 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-12-09T09:09:04.450 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:09:04.488 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:04 +0000 (0:00:00.194) 0:04:51.771 ******* 2019-12-09T09:09:05.249 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-09T09:09:05.294 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:05 +0000 (0:00:00.805) 0:04:52.577 ******* 2019-12-09T09:09:05.321 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2019-12-09T09:09:05.323 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:05.369 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:05 +0000 (0:00:00.074) 0:04:52.652 ******* 2019-12-09T09:09:05.397 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2019-12-09T09:09:05.399 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:05.440 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:05 +0000 (0:00:00.071) 0:04:52.723 ******* 2019-12-09T09:09:05.448 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2019-12-09T09:09:05.449 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-09T09:09:05.490 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:05 +0000 (0:00:00.049) 0:04:52.773 ******* 2019-12-09T09:09:05.548 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:05.596 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:05 +0000 (0:00:00.105) 0:04:52.878 ******* 2019-12-09T09:09:05.623 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2019-12-09T09:09:05.624 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:05.688 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:05 +0000 (0:00:00.092) 0:04:52.971 ******* 2019-12-09T09:09:05.795 INFO:teuthology.task.ansible.out: 2019-12-09T09:09:05.797 INFO:teuthology.task.ansible.out:TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2019-12-09T09:09:05.798 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-09T09:09:05.843 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:05 +0000 (0:00:00.155) 0:04:53.126 ******* 2019-12-09T09:09:06.042 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-09T09:09:06.095 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:06 +0000 (0:00:00.251) 0:04:53.378 ******* 2019-12-09T09:09:06.901 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2019-12-09T09:09:06.903 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-09T09:09:06.951 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:06 +0000 (0:00:00.856) 0:04:54.234 ******* 2019-12-09T09:09:07.124 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-09T09:09:07.222 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:07 +0000 (0:00:00.271) 0:04:54.505 ******* 2019-12-09T09:09:08.744 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2019-12-09T09:09:08.745 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-09T09:09:08.783 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:08 +0000 (0:00:01.560) 0:04:56.066 ******* 2019-12-09T09:09:08.792 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-09T09:09:08.836 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:08 +0000 (0:00:00.053) 0:04:56.119 ******* 2019-12-09T09:09:08.843 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-12-09T09:09:08.845 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:08.884 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:08 +0000 (0:00:00.047) 0:04:56.166 ******* 2019-12-09T09:09:08.892 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:08.934 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:08 +0000 (0:00:00.050) 0:04:56.217 ******* 2019-12-09T09:09:08.944 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi022.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2019-12-09T09:09:08.995 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:08 +0000 (0:00:00.060) 0:04:56.278 ******* 2019-12-09T09:09:09.005 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2019-12-09T09:09:09.052 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:09 +0000 (0:00:00.057) 0:04:56.335 ******* 2019-12-09T09:09:09.341 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-12-09T09:09:09.343 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-09T09:09:09.391 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:09 +0000 (0:00:00.338) 0:04:56.674 ******* 2019-12-09T09:09:09.412 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-09T09:09:09.465 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:09 +0000 (0:00:00.073) 0:04:56.747 ******* 2019-12-09T09:09:09.473 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-12-09T09:09:09.475 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:09.523 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:09 +0000 (0:00:00.058) 0:04:56.806 ******* 2019-12-09T09:09:09.533 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:09.575 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:09 +0000 (0:00:00.052) 0:04:56.858 ******* 2019-12-09T09:09:09.581 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-12-09T09:09:09.582 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-09T09:09:09.625 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:09 +0000 (0:00:00.048) 0:04:56.907 ******* 2019-12-09T09:09:09.633 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-09T09:09:09.670 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:09 +0000 (0:00:00.046) 0:04:56.953 ******* 2019-12-09T09:09:09.676 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2019-12-09T09:09:09.677 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:09.719 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:09 +0000 (0:00:00.046) 0:04:57.000 ******* 2019-12-09T09:09:09.723 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:09.777 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:09 +0000 (0:00:00.059) 0:04:57.060 ******* 2019-12-09T09:09:10.048 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2019-12-09T09:09:10.049 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-09T09:09:10.106 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:10 +0000 (0:00:00.322) 0:04:57.383 ******* 2019-12-09T09:09:10.118 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-09T09:09:10.158 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:10 +0000 (0:00:00.056) 0:04:57.439 ******* 2019-12-09T09:09:10.445 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2019-12-09T09:09:10.447 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-09T09:09:10.483 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:10 +0000 (0:00:00.326) 0:04:57.766 ******* 2019-12-09T09:09:10.501 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-09T09:09:10.540 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:10 +0000 (0:00:00.056) 0:04:57.822 ******* 2019-12-09T09:09:10.546 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-12-09T09:09:10.548 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.588 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:10 +0000 (0:00:00.047) 0:04:57.870 ******* 2019-12-09T09:09:10.597 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.642 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:10 +0000 (0:00:00.054) 0:04:57.925 ******* 2019-12-09T09:09:10.651 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2019-12-09T09:09:10.653 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} skipping: [smithi022.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.654 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi022.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.674 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.729 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:10 +0000 (0:00:00.087) 0:04:58.012 ******* 2019-12-09T09:09:10.736 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.737 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.738 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.740 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi035.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.796 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:10 +0000 (0:00:00.066) 0:04:58.078 ******* 2019-12-09T09:09:10.802 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-12-09T09:09:10.803 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.805 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.807 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.808 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.810 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.852 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:10 +0000 (0:00:00.057) 0:04:58.135 ******* 2019-12-09T09:09:10.863 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi035.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.898 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.899 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.901 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.903 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:10.954 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:10 +0000 (0:00:00.101) 0:04:58.237 ******* 2019-12-09T09:09:10.995 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:10 +0000 (0:00:00.041) 0:04:58.278 ******* 2019-12-09T09:09:11.003 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-12-09T09:09:11.005 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.059 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.063) 0:04:58.342 ******* 2019-12-09T09:09:11.071 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:09:11.073 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.122 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.062) 0:04:58.405 ******* 2019-12-09T09:09:11.128 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-12-09T09:09:11.130 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.177 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.055) 0:04:58.460 ******* 2019-12-09T09:09:11.185 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-12-09T09:09:11.187 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.230 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.052) 0:04:58.513 ******* 2019-12-09T09:09:11.238 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-12-09T09:09:11.239 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.292 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.061) 0:04:58.575 ******* 2019-12-09T09:09:11.299 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-12-09T09:09:11.301 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.353 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.060) 0:04:58.635 ******* 2019-12-09T09:09:11.364 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.410 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.056) 0:04:58.692 ******* 2019-12-09T09:09:11.418 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-12-09T09:09:11.420 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.458 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.048) 0:04:58.741 ******* 2019-12-09T09:09:11.464 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-09T09:09:11.466 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.509 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.051) 0:04:58.792 ******* 2019-12-09T09:09:11.516 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2019-12-09T09:09:11.516 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.559 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.050) 0:04:58.842 ******* 2019-12-09T09:09:11.572 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.608 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.048) 0:04:58.891 ******* 2019-12-09T09:09:11.615 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-09T09:09:11.617 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.658 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.050) 0:04:58.941 ******* 2019-12-09T09:09:11.664 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-09T09:09:11.667 INFO:teuthology.task.ansible.out:skipping: [smithi022.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.725 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.062) 0:04:59.004 ******* 2019-12-09T09:09:11.729 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2019-12-09T09:09:11.731 INFO:teuthology.task.ansible.out:skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.776 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.054) 0:04:59.058 ******* 2019-12-09T09:09:11.816 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi035.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:09:11.855 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:11 +0000 (0:00:00.079) 0:04:59.138 ******* 2019-12-09T09:09:12.091 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T09:09:12.092 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.016339", "end": "2019-12-09 09:09:12.055405", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-09 09:09:12.039066", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:09:12.199 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011942", "end": "2019-12-09 09:09:12.164068", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-09 09:09:12.152126", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:09:12.424 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006360", "end": "2019-12-09 09:09:12.370050", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-09 09:09:12.363690", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:09:12.532 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.007202", "end": "2019-12-09 09:09:12.492921", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-09 09:09:12.485719", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:09:12.739 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.006876", "end": "2019-12-09 09:09:12.706522", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:09:12.699646", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-09T09:09:12.866 INFO:teuthology.task.ansible.out:ok: [smithi035.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.007540", "end": "2019-12-09 09:09:12.837082", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:09:12.829542", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-09T09:09:13.067 INFO:teuthology.task.ansible.out:ok: [smithi022.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.007652", "end": "2019-12-09 09:09:13.019795", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-09 09:09:13.012143", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:09:13.118 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:13 +0000 (0:00:01.263) 0:05:00.401 ******* 2019-12-09T09:09:13.146 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-09T09:09:13.187 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:13 +0000 (0:00:00.068) 0:05:00.470 ******* 2019-12-09T09:09:13.236 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi035.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.008581", "end": "2019-12-09 09:09:13.149335", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-09 09:09:13.140754", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:09:13.282 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:13 +0000 (0:00:00.094) 0:05:00.564 ******* 2019-12-09T09:09:13.309 INFO:teuthology.task.ansible.out: 2019-12-09T09:09:13.311 INFO:teuthology.task.ansible.out:TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-09T09:09:13.352 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:09:13 +0000 (0:00:00.070) 0:05:00.635 ******* 2019-12-09T09:09:13.354 INFO:teuthology.task.ansible.out: 2019-12-09T09:09:13.537 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2019-12-09T09:09:13.538 INFO:teuthology.task.ansible.out:changed: [smithi022.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:09:13.683 INFO:teuthology.task.ansible.out:changed: [smithi035.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:09:13.816 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2019-12-09T09:09:13.819 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2019-12-09T09:09:13.821 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2019-12-09T09:09:13.823 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi022.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=160 rescued=0 ignored=3 smithi035.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=160 rescued=0 ignored=3 Monday 09 December 2019 09:09:13 +0000 (0:00:00.469) 0:05:01.104 ******* =============================================================================== users : Create all admin users with sudo access. ----------------------- 36.94s users : Update authorized_keys using the keys repo --------------------- 35.72s common : nrpe - Ensure SELinux policy is up to date -------------------- 27.43s testnode : Install packages -------------------------------------------- 17.94s users : Remove revoked users ------------------------------------------- 16.41s common : Update yum cache ---------------------------------------------- 12.36s common : nrpe - Load SELinux policy package ----------------------------- 8.89s 2019-12-09T09:09:13.824 INFO:teuthology.task.ansible.out:common : nrpe - Remove SELinux policy package --------------------------- 8.79s testnode : Ensure ceph packages are not present. ------------------------ 8.25s testnode : Zap all non-root disks --------------------------------------- 7.25s common : Make sure smartmontools is installed --------------------------- 3.05s ansible-managed : Add authorized keys for the ansible user. ------------- 2.81s testnode : Ensure ceph dependency packages are not present. ------------- 2.76s Gathering Facts --------------------------------------------------------- 2.70s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 2.64s testnode : Rebuild rpmdb ------------------------------------------------ 2.32s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 2.04s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 1.89s testnode : Configure version specific repos in /etc/yum.repos.d/ -------- 1.75s testnode : Create logical volume(s) ------------------------------------- 1.60s 2019-12-09T09:09:14.095 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi035.front.sepia.ceph.com'), Remote(name='ubuntu@smithi022.front.sepia.ceph.com')] 2019-12-09T09:09:14.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:09:14.098 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi035.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:09:14.199 INFO:teuthology.orchestra.run.smithi035:> true 2019-12-09T09:09:14.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:09:14.600 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi022.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:09:14.699 INFO:teuthology.orchestra.run.smithi022:> true 2019-12-09T09:09:15.097 INFO:teuthology.run_tasks:Running task clock... 2019-12-09T09:09:15.108 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2019-12-09T09:09:15.109 INFO:teuthology.orchestra.run.smithi035:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-09T09:09:15.257 INFO:teuthology.orchestra.run.smithi035.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-09T09:09:25.348 INFO:teuthology.orchestra.run.smithi035.stdout:ntpd: time slew +0.000541s 2019-12-09T09:09:25.388 INFO:teuthology.orchestra.run.smithi035.stderr:Failed to start ntp.service: Unit not found. 2019-12-09T09:09:25.464 INFO:teuthology.orchestra.run.smithi035.stdout: remote refid st t when poll reach delay offset jitter 2019-12-09T09:09:25.464 INFO:teuthology.orchestra.run.smithi035.stdout:============================================================================== 2019-12-09T09:09:25.464 INFO:teuthology.orchestra.run.smithi035.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:09:25.465 INFO:teuthology.orchestra.run.smithi035.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:09:25.465 INFO:teuthology.orchestra.run.smithi035.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:09:25.465 INFO:teuthology.orchestra.run.smithi035.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:09:25.466 INFO:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-09T09:09:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-09T09:09:35.705 INFO:teuthology.orchestra.run.smithi022.stdout:ntpd: time slew -0.001516s 2019-12-09T09:09:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit not found. 2019-12-09T09:09:35.830 INFO:teuthology.orchestra.run.smithi022.stdout: remote refid st t when poll reach delay offset jitter 2019-12-09T09:09:35.831 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================== 2019-12-09T09:09:35.831 INFO:teuthology.orchestra.run.smithi022.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:09:35.831 INFO:teuthology.orchestra.run.smithi022.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:09:35.831 INFO:teuthology.orchestra.run.smithi022.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:09:35.832 INFO:teuthology.orchestra.run.smithi022.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:09:35.832 INFO:teuthology.run_tasks:Running task install... 2019-12-09T09:09:35.852 DEBUG:teuthology.task.install:project ceph 2019-12-09T09:09:35.853 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}} 2019-12-09T09:09:35.853 DEBUG:teuthology.task.install:config {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'} 2019-12-09T09:09:35.853 INFO:teuthology.task.install:Using flavor: notcmalloc 2019-12-09T09:09:35.865 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-daemon', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python-ceph', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-daemon', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-ssh', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python-ceph', 'rbd-fuse']} 2019-12-09T09:09:35.865 INFO:teuthology.task.install:extra packages: [] 2019-12-09T09:09:35.866 INFO:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2019-12-09T09:09:36.025 INFO:teuthology.orchestra.run.smithi035.stdout:CentOS 2019-12-09T09:09:36.028 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:09:36.028 INFO:teuthology.orchestra.run.smithi022:> sudo lsb_release -is 2019-12-09T09:09:36.029 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-12-09T09:09:36.029 INFO:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2019-12-09T09:09:36.164 INFO:teuthology.orchestra.run.smithi035.stdout:CentOS 2019-12-09T09:09:36.179 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:09:36.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:09:36.195 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS 2019-12-09T09:09:36.197 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:09:36.198 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-12-09T09:09:36.198 INFO:teuthology.orchestra.run.smithi022:> sudo lsb_release -is 2019-12-09T09:09:36.346 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-ofriedma-dynamic-disable/1351c819e1a9518cde4c8b12d645782009028360/centos/7/flavors/notcmalloc/ 2019-12-09T09:09:36.347 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7900.g1351c81 2019-12-09T09:09:36.352 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS 2019-12-09T09:09:36.353 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:09:36.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:09:36.500 INFO:teuthology.orchestra.run.smithi035:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-09T09:09:36.550 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-ofriedma-dynamic-disable/1351c819e1a9518cde4c8b12d645782009028360/centos/7/flavors/notcmalloc/ 2019-12-09T09:09:36.551 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7900.g1351c81 2019-12-09T09:09:36.624 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-12-09T09:09:36.625 INFO:teuthology.orchestra.run.smithi035:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1351c819e1a9518cde4c8b12d645782009028360/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-09T09:09:36.678 INFO:teuthology.orchestra.run.smithi022:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-09T09:09:36.729 INFO:teuthology.orchestra.run.smithi035:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-09T09:09:36.810 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-12-09T09:09:36.810 INFO:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1351c819e1a9518cde4c8b12d645782009028360/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-09T09:09:36.831 INFO:teuthology.orchestra.run.smithi035:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-09T09:09:36.923 INFO:teuthology.orchestra.run.smithi022:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-09T09:09:36.928 INFO:teuthology.orchestra.run.smithi035.stdout:check_obsoletes = 1 2019-12-09T09:09:36.929 INFO:teuthology.orchestra.run.smithi035:> sudo yum clean all 2019-12-09T09:09:37.031 INFO:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-09T09:09:37.141 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2019-12-09T09:09:37.144 INFO:teuthology.orchestra.run.smithi022:> sudo yum clean all 2019-12-09T09:09:37.154 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:09:37.205 INFO:teuthology.orchestra.run.smithi035.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-09T09:09:37.205 INFO:teuthology.orchestra.run.smithi035.stdout: : lab-extras updates 2019-12-09T09:09:37.241 INFO:teuthology.orchestra.run.smithi035.stdout:Cleaning up list of fastest mirrors 2019-12-09T09:09:37.241 INFO:teuthology.orchestra.run.smithi035.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-09T09:09:37.255 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-radosgw 2019-12-09T09:09:37.384 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:09:37.429 INFO:teuthology.orchestra.run.smithi022.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-09T09:09:37.429 INFO:teuthology.orchestra.run.smithi022.stdout: : lab-extras updates 2019-12-09T09:09:37.476 INFO:teuthology.orchestra.run.smithi022.stdout:Cleaning up list of fastest mirrors 2019-12-09T09:09:37.478 INFO:teuthology.orchestra.run.smithi022.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-09T09:09:37.506 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw 2019-12-09T09:09:37.517 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:09:37.558 INFO:teuthology.orchestra.run.smithi035.stdout:Determining fastest mirrors 2019-12-09T09:09:37.673 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:09:37.712 INFO:teuthology.orchestra.run.smithi022.stdout:Determining fastest mirrors 2019-12-09T09:09:37.890 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:09:37.891 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:09:37.891 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:09:37.892 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:09:38.309 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:09:38.310 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:09:38.310 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:09:38.311 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:09:45.601 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:09:45.965 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:09:45.966 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:09:45.966 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:45.980 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.091 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.092 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.093 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.095 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.095 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.097 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.098 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.099 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.101 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:09:46.101 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.110 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.111 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.112 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.114 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.115 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.116 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.118 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-12-09T09:09:46.119 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-12-09T09:09:46.119 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:09:46.120 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package librados2.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:09:46.122 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-12-09T09:09:46.123 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.124 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:09:46.124 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.129 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.130 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.131 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.132 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.133 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.134 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.135 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.136 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.137 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.137 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.138 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.138 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-09T09:09:46.138 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.139 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:09:46.139 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:09:46.140 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-09T09:09:46.141 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:09:46.142 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:09:46.142 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:09:46.142 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-09T09:09:46.144 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.145 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.145 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.145 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.146 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-09T09:09:46.146 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.146 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.147 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.148 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-09T09:09:46.258 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:09:46.625 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:09:46.625 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:09:46.625 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.639 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.748 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.750 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.751 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.753 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.753 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.755 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.755 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.757 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.758 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:09:46.758 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.768 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.769 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.770 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.771 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.772 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.773 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.776 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-12-09T09:09:46.777 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-12-09T09:09:46.777 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:09:46.778 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package librados2.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:09:46.779 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-12-09T09:09:46.780 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.782 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:09:46.782 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.787 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.788 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.789 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.790 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.791 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.792 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.793 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.794 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.794 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.795 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.796 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:09:46.797 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-09T09:09:46.797 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.797 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:09:46.798 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:09:46.799 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-09T09:09:46.799 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:09:46.800 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:09:46.800 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:09:46.800 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-09T09:09:46.801 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.802 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.802 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.803 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.804 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-09T09:09:46.804 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.805 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.805 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:09:46.805 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-09T09:09:47.061 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:09:47.151 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:09:47.151 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:09:47.155 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:09:47.155 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:09:47.155 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository 2019-12-09T09:09:47.155 INFO:teuthology.orchestra.run.smithi035.stdout: Size 2019-12-09T09:09:47.155 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:09:47.156 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:09:47.156 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-radosgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 6.2 M 2019-12-09T09:09:47.156 INFO:teuthology.orchestra.run.smithi035.stdout:Installing for dependencies: 2019-12-09T09:09:47.156 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-base x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.6 M 2019-12-09T09:09:47.156 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 M 2019-12-09T09:09:47.156 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-selinux x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 k 2019-12-09T09:09:47.156 INFO:teuthology.orchestra.run.smithi035.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-09T09:09:47.157 INFO:teuthology.orchestra.run.smithi035.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-09T09:09:47.157 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 489 k 2019-12-09T09:09:47.157 INFO:teuthology.orchestra.run.smithi035.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-12-09T09:09:47.157 INFO:teuthology.orchestra.run.smithi035.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-12-09T09:09:47.157 INFO:teuthology.orchestra.run.smithi035.stdout: libradosstriper1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 374 k 2019-12-09T09:09:47.157 INFO:teuthology.orchestra.run.smithi035.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-12-09T09:09:47.157 INFO:teuthology.orchestra.run.smithi035.stdout: librgw2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.4 M 2019-12-09T09:09:47.158 INFO:teuthology.orchestra.run.smithi035.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-09T09:09:47.158 INFO:teuthology.orchestra.run.smithi035.stdout: python-ceph-argparse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 35 k 2019-12-09T09:09:47.158 INFO:teuthology.orchestra.run.smithi035.stdout: python-ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 42 k 2019-12-09T09:09:47.158 INFO:teuthology.orchestra.run.smithi035.stdout: python-cephfs x86_64 2:15.0.0-7900.g1351c81.el7 ceph 102 k 2019-12-09T09:09:47.158 INFO:teuthology.orchestra.run.smithi035.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-09T09:09:47.158 INFO:teuthology.orchestra.run.smithi035.stdout: python-rados x86_64 2:15.0.0-7900.g1351c81.el7 ceph 209 k 2019-12-09T09:09:47.158 INFO:teuthology.orchestra.run.smithi035.stdout: python-rbd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 199 k 2019-12-09T09:09:47.159 INFO:teuthology.orchestra.run.smithi035.stdout: python-rgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 77 k 2019-12-09T09:09:47.159 INFO:teuthology.orchestra.run.smithi035.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-09T09:09:47.159 INFO:teuthology.orchestra.run.smithi035.stdout:Updating for dependencies: 2019-12-09T09:09:47.159 INFO:teuthology.orchestra.run.smithi035.stdout: librados2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 3.5 M 2019-12-09T09:09:47.159 INFO:teuthology.orchestra.run.smithi035.stdout: librbd1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.8 M 2019-12-09T09:09:47.159 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:09:47.159 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:09:47.159 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:09:47.160 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package (+20 Dependent packages) 2019-12-09T09:09:47.160 INFO:teuthology.orchestra.run.smithi035.stdout:Upgrade ( 2 Dependent packages) 2019-12-09T09:09:47.160 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:09:47.160 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 45 M 2019-12-09T09:09:47.160 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:09:47.161 INFO:teuthology.orchestra.run.smithi035.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:09:47.711 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:09:47.800 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:09:47.800 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:09:47.804 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:09:47.804 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:09:47.804 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository 2019-12-09T09:09:47.805 INFO:teuthology.orchestra.run.smithi022.stdout: Size 2019-12-09T09:09:47.805 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:09:47.805 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:09:47.805 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 6.2 M 2019-12-09T09:09:47.805 INFO:teuthology.orchestra.run.smithi022.stdout:Installing for dependencies: 2019-12-09T09:09:47.805 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.6 M 2019-12-09T09:09:47.805 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 M 2019-12-09T09:09:47.805 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 k 2019-12-09T09:09:47.806 INFO:teuthology.orchestra.run.smithi022.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-09T09:09:47.806 INFO:teuthology.orchestra.run.smithi022.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-09T09:09:47.806 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 489 k 2019-12-09T09:09:47.806 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-12-09T09:09:47.806 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-12-09T09:09:47.806 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 374 k 2019-12-09T09:09:47.806 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-12-09T09:09:47.807 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.4 M 2019-12-09T09:09:47.807 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-09T09:09:47.807 INFO:teuthology.orchestra.run.smithi022.stdout: python-ceph-argparse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 35 k 2019-12-09T09:09:47.807 INFO:teuthology.orchestra.run.smithi022.stdout: python-ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 42 k 2019-12-09T09:09:47.807 INFO:teuthology.orchestra.run.smithi022.stdout: python-cephfs x86_64 2:15.0.0-7900.g1351c81.el7 ceph 102 k 2019-12-09T09:09:47.807 INFO:teuthology.orchestra.run.smithi022.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-09T09:09:47.807 INFO:teuthology.orchestra.run.smithi022.stdout: python-rados x86_64 2:15.0.0-7900.g1351c81.el7 ceph 209 k 2019-12-09T09:09:47.808 INFO:teuthology.orchestra.run.smithi022.stdout: python-rbd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 199 k 2019-12-09T09:09:47.808 INFO:teuthology.orchestra.run.smithi022.stdout: python-rgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 77 k 2019-12-09T09:09:47.808 INFO:teuthology.orchestra.run.smithi022.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-09T09:09:47.808 INFO:teuthology.orchestra.run.smithi022.stdout:Updating for dependencies: 2019-12-09T09:09:47.808 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 3.5 M 2019-12-09T09:09:47.808 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.8 M 2019-12-09T09:09:47.808 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:09:47.809 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:09:47.809 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:09:47.809 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package (+20 Dependent packages) 2019-12-09T09:09:47.809 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade ( 2 Dependent packages) 2019-12-09T09:09:47.809 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:09:47.809 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 45 M 2019-12-09T09:09:47.809 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:09:47.810 INFO:teuthology.orchestra.run.smithi022.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:09:49.604 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:09:49.604 INFO:teuthology.orchestra.run.smithi035.stdout:Total 18 MB/s | 45 MB 00:02 2019-12-09T09:09:49.618 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:09:49.680 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:09:50.078 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:09:50.079 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:09:50.802 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-12-09T09:09:51.184 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-12-09T09:09:51.572 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-12-09T09:09:51.883 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:09:52.114 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:09:52.115 INFO:teuthology.orchestra.run.smithi022.stdout:Total 10 MB/s | 45 MB 00:04 2019-12-09T09:09:52.132 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:09:52.166 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:09:52.209 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:09:52.409 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:09:52.409 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:09:52.531 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-12-09T09:09:53.041 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-12-09T09:09:53.346 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-12-09T09:09:53.378 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-12-09T09:09:53.744 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-12-09T09:09:53.836 INFO:teuthology.orchestra.run.smithi035.stdout: Updating : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 8/25 2019-12-09T09:09:54.113 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:09:54.354 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:09:54.713 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-12-09T09:09:54.715 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:09:55.078 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:09:55.489 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-12-09T09:09:55.672 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 11/25 2019-12-09T09:09:55.826 INFO:teuthology.orchestra.run.smithi022.stdout: Updating : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 8/25 2019-12-09T09:09:56.016 INFO:teuthology.orchestra.run.smithi035.stdout: Updating : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:09:56.444 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 13/25 2019-12-09T09:09:56.735 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 14/25 2019-12-09T09:09:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:09:57.096 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:09:57.159 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:09:57.401 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:09:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 11/25 2019-12-09T09:09:57.726 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-12-09T09:09:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: Updating : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:09:58.114 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/25 2019-12-09T09:09:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 13/25 2019-12-09T09:09:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 14/25 2019-12-09T09:09:58.917 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:09:59.199 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:09:59.480 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-12-09T09:09:59.810 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/25 2019-12-09T09:10:00.604 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 19/25 2019-12-09T09:10:01.802 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:10:02.240 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 19/25 2019-12-09T09:10:02.344 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 21/25 2019-12-09T09:10:03.425 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:10:03.943 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 21/25 2019-12-09T09:10:12.790 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:10:13.255 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:10:13.639 INFO:teuthology.orchestra.run.smithi035.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:10:14.117 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:10:14.504 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:10:14.736 INFO:teuthology.orchestra.run.smithi035.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:10:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:10:14.869 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 1/25 2019-12-09T09:10:15.002 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-12-09T09:10:15.089 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:10:15.154 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 3/25 2019-12-09T09:10:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 1/25 2019-12-09T09:10:15.305 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:10:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-12-09T09:10:15.388 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:10:15.444 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 3/25 2019-12-09T09:10:15.530 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 6/25 2019-12-09T09:10:15.544 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:10:15.638 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 7/25 2019-12-09T09:10:15.644 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:10:15.739 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 8/25 2019-12-09T09:10:15.745 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 6/25 2019-12-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:10:15.845 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 7/25 2019-12-09T09:10:15.940 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:10:15.955 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 8/25 2019-12-09T09:10:16.055 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:10:16.057 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 11/25 2019-12-09T09:10:16.156 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:10:16.200 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:10:16.281 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 11/25 2019-12-09T09:10:16.316 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 13/25 2019-12-09T09:10:16.399 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 14/25 2019-12-09T09:10:16.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 13/25 2019-12-09T09:10:16.568 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:10:16.616 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 14/25 2019-12-09T09:10:16.684 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:10:16.719 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:10:16.817 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:10:16.844 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 17/25 2019-12-09T09:10:16.926 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 17/25 2019-12-09T09:10:16.961 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 18/25 2019-12-09T09:10:17.053 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 18/25 2019-12-09T09:10:17.071 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 19/25 2019-12-09T09:10:17.237 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:10:17.265 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 19/25 2019-12-09T09:10:17.348 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-12-09T09:10:17.465 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:10:17.572 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:10:17.573 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:10:17.770 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:10:17.880 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-12-09T09:10:17.914 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:10:17.914 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:17.914 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:10:17.914 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.915 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:17.915 INFO:teuthology.orchestra.run.smithi035.stdout:Dependency Installed: 2019-12-09T09:10:17.915 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.915 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.915 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.915 INFO:teuthology.orchestra.run.smithi035.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-09T09:10:17.915 INFO:teuthology.orchestra.run.smithi035.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-09T09:10:17.916 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.916 INFO:teuthology.orchestra.run.smithi035.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-12-09T09:10:17.916 INFO:teuthology.orchestra.run.smithi035.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-12-09T09:10:17.916 INFO:teuthology.orchestra.run.smithi035.stdout: libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.916 INFO:teuthology.orchestra.run.smithi035.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-12-09T09:10:17.917 INFO:teuthology.orchestra.run.smithi035.stdout: librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.917 INFO:teuthology.orchestra.run.smithi035.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-09T09:10:17.917 INFO:teuthology.orchestra.run.smithi035.stdout: python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.917 INFO:teuthology.orchestra.run.smithi035.stdout: python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.917 INFO:teuthology.orchestra.run.smithi035.stdout: python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.917 INFO:teuthology.orchestra.run.smithi035.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-09T09:10:17.917 INFO:teuthology.orchestra.run.smithi035.stdout: python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.918 INFO:teuthology.orchestra.run.smithi035.stdout: python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.918 INFO:teuthology.orchestra.run.smithi035.stdout: python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.918 INFO:teuthology.orchestra.run.smithi035.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-09T09:10:17.918 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:17.918 INFO:teuthology.orchestra.run.smithi035.stdout:Dependency Updated: 2019-12-09T09:10:17.919 INFO:teuthology.orchestra.run.smithi035.stdout: librados2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.919 INFO:teuthology.orchestra.run.smithi035.stdout: librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:17.919 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:17.919 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:10:17.980 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:10:18.038 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-test 2019-12-09T09:10:18.089 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:10:18.105 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:10:18.346 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:10:18.393 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:10:18.393 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:10:18.393 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:10:18.394 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:10:18.394 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:10:18.472 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:10:18.472 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:18.473 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:10:18.473 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.473 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:18.473 INFO:teuthology.orchestra.run.smithi022.stdout:Dependency Installed: 2019-12-09T09:10:18.474 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.474 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.474 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.474 INFO:teuthology.orchestra.run.smithi022.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-09T09:10:18.474 INFO:teuthology.orchestra.run.smithi022.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-09T09:10:18.475 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.475 INFO:teuthology.orchestra.run.smithi022.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-12-09T09:10:18.475 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-12-09T09:10:18.475 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.475 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-12-09T09:10:18.476 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.476 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-09T09:10:18.478 INFO:teuthology.orchestra.run.smithi022.stdout: python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.478 INFO:teuthology.orchestra.run.smithi022.stdout: python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.479 INFO:teuthology.orchestra.run.smithi022.stdout: python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.479 INFO:teuthology.orchestra.run.smithi022.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-09T09:10:18.479 INFO:teuthology.orchestra.run.smithi022.stdout: python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.480 INFO:teuthology.orchestra.run.smithi022.stdout: python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.480 INFO:teuthology.orchestra.run.smithi022.stdout: python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.480 INFO:teuthology.orchestra.run.smithi022.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-09T09:10:18.480 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:18.481 INFO:teuthology.orchestra.run.smithi022.stdout:Dependency Updated: 2019-12-09T09:10:18.481 INFO:teuthology.orchestra.run.smithi022.stdout: librados2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.481 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:18.481 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:18.482 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:10:18.604 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-test 2019-12-09T09:10:18.933 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:10:18.977 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:10:18.977 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:10:18.977 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:10:18.978 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:10:18.978 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:10:19.066 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:10:19.415 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:10:19.415 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:19.416 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:19.436 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:19.466 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:19.467 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:19.467 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:19.468 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-09T09:10:19.468 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-09T09:10:19.471 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-09T09:10:19.471 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-09T09:10:19.474 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:19.474 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-09T09:10:19.658 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:10:19.861 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:10:20.009 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:10:20.009 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:20.010 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:20.036 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:20.059 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:20.060 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:20.062 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:20.062 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-09T09:10:20.063 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-09T09:10:20.064 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-09T09:10:20.065 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-09T09:10:20.068 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:20.068 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-09T09:10:20.105 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:20.105 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:10:20.105 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:20.105 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:20.106 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository 2019-12-09T09:10:20.106 INFO:teuthology.orchestra.run.smithi035.stdout: Size 2019-12-09T09:10:20.106 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:20.106 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:10:20.107 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-test x86_64 2:15.0.0-7900.g1351c81.el7 ceph 38 M 2019-12-09T09:10:20.107 INFO:teuthology.orchestra.run.smithi035.stdout:Installing for dependencies: 2019-12-09T09:10:20.107 INFO:teuthology.orchestra.run.smithi035.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-09T09:10:20.107 INFO:teuthology.orchestra.run.smithi035.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-09T09:10:20.108 INFO:teuthology.orchestra.run.smithi035.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-09T09:10:20.108 INFO:teuthology.orchestra.run.smithi035.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-09T09:10:20.108 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:20.108 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:10:20.109 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:20.109 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:10:20.109 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:20.109 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 38 M 2019-12-09T09:10:20.109 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 134 M 2019-12-09T09:10:20.110 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:10:20.451 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:10:20.676 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:20.676 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:10:20.678 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:20.678 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:20.679 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository 2019-12-09T09:10:20.679 INFO:teuthology.orchestra.run.smithi022.stdout: Size 2019-12-09T09:10:20.679 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:20.679 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:10:20.679 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:15.0.0-7900.g1351c81.el7 ceph 38 M 2019-12-09T09:10:20.680 INFO:teuthology.orchestra.run.smithi022.stdout:Installing for dependencies: 2019-12-09T09:10:20.680 INFO:teuthology.orchestra.run.smithi022.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-09T09:10:20.680 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-09T09:10:20.680 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-09T09:10:20.681 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-09T09:10:20.681 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:20.681 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:10:20.681 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:20.681 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:10:20.682 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:20.682 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 38 M 2019-12-09T09:10:20.682 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 134 M 2019-12-09T09:10:20.682 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:10:21.936 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:10:21.936 INFO:teuthology.orchestra.run.smithi035.stdout:Total 21 MB/s | 38 MB 00:01 2019-12-09T09:10:21.938 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:10:21.957 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:10:22.017 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:10:22.017 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:10:22.724 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-09T09:10:23.018 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-09T09:10:23.353 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-09T09:10:27.461 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:10:27.461 INFO:teuthology.orchestra.run.smithi022.stdout:Total 5.6 MB/s | 38 MB 00:06 2019-12-09T09:10:27.464 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:10:27.488 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:10:27.572 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:10:27.597 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:10:27.598 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:10:27.889 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:10:27.980 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:10:28.214 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-12-09T09:10:28.322 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-12-09T09:10:28.407 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-09T09:10:28.455 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:10:28.668 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-09T09:10:28.955 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-09T09:10:28.957 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-09T09:10:28.958 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:28.958 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:10:28.959 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:28.959 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:28.968 INFO:teuthology.orchestra.run.smithi035.stdout:Dependency Installed: 2019-12-09T09:10:28.977 INFO:teuthology.orchestra.run.smithi035.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-09T09:10:28.978 INFO:teuthology.orchestra.run.smithi035.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-09T09:10:28.978 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:28.978 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:10:29.090 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph 2019-12-09T09:10:29.381 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:10:29.425 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:10:29.425 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:10:29.426 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:10:29.426 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:10:29.426 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:10:30.112 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:10:30.470 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:10:30.472 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:30.472 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:30.473 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:30.504 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:30.505 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:30.507 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:30.507 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:30.508 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:30.521 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:30.523 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:30.524 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:30.524 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:30.524 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:30.525 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:30.527 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:30.527 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:30.537 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:30.542 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:30.543 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-09T09:10:30.543 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-09T09:10:30.543 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-09T09:10:30.543 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:10:30.543 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:10:30.543 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:10:30.544 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:10:30.552 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:10:30.552 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-12-09T09:10:30.552 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-12-09T09:10:30.553 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-09T09:10:30.553 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-12-09T09:10:30.553 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-12-09T09:10:30.553 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:30.553 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-09T09:10:30.553 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:10:30.553 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:10:30.573 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-09T09:10:30.574 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-09T09:10:30.579 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-09T09:10:30.579 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-09T09:10:30.580 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-09T09:10:30.580 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-12-09T09:10:30.580 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:30.580 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-09T09:10:30.580 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-09T09:10:30.580 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-09T09:10:30.580 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:30.581 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-09T09:10:30.581 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-09T09:10:30.581 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:30.581 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-09T09:10:30.928 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:10:31.157 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:31.157 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:10:31.164 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:31.164 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:31.165 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository 2019-12-09T09:10:31.165 INFO:teuthology.orchestra.run.smithi035.stdout: Size 2019-12-09T09:10:31.165 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:31.165 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:10:31.166 INFO:teuthology.orchestra.run.smithi035.stdout: ceph x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.0 k 2019-12-09T09:10:31.166 INFO:teuthology.orchestra.run.smithi035.stdout:Installing for dependencies: 2019-12-09T09:10:31.166 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mds x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.9 M 2019-12-09T09:10:31.166 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.7 M 2019-12-09T09:10:31.166 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 4.4 M 2019-12-09T09:10:31.166 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-osd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 16 M 2019-12-09T09:10:31.167 INFO:teuthology.orchestra.run.smithi035.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-09T09:10:31.167 INFO:teuthology.orchestra.run.smithi035.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-09T09:10:31.168 INFO:teuthology.orchestra.run.smithi035.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-09T09:10:31.168 INFO:teuthology.orchestra.run.smithi035.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-09T09:10:31.168 INFO:teuthology.orchestra.run.smithi035.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-09T09:10:31.168 INFO:teuthology.orchestra.run.smithi035.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-09T09:10:31.168 INFO:teuthology.orchestra.run.smithi035.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-09T09:10:31.168 INFO:teuthology.orchestra.run.smithi035.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-12-09T09:10:31.169 INFO:teuthology.orchestra.run.smithi035.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-09T09:10:31.169 INFO:teuthology.orchestra.run.smithi035.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-09T09:10:31.169 INFO:teuthology.orchestra.run.smithi035.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-09T09:10:31.169 INFO:teuthology.orchestra.run.smithi035.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-09T09:10:31.169 INFO:teuthology.orchestra.run.smithi035.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-09T09:10:31.169 INFO:teuthology.orchestra.run.smithi035.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-09T09:10:31.169 INFO:teuthology.orchestra.run.smithi035.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-09T09:10:31.170 INFO:teuthology.orchestra.run.smithi035.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-12-09T09:10:31.170 INFO:teuthology.orchestra.run.smithi035.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-12-09T09:10:31.170 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:31.170 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:10:31.170 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:31.170 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package (+21 Dependent packages) 2019-12-09T09:10:31.170 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:31.171 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 28 M 2019-12-09T09:10:31.171 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 87 M 2019-12-09T09:10:31.171 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:10:32.620 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:10:32.620 INFO:teuthology.orchestra.run.smithi035.stdout:Total 19 MB/s | 28 MB 00:01 2019-12-09T09:10:32.645 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:10:32.680 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:10:32.949 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:10:32.950 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:10:33.130 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:10:33.465 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:10:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:10:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-12-09T09:10:33.766 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-12-09T09:10:33.806 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-12-09T09:10:33.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:10:34.101 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-12-09T09:10:34.317 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-09T09:10:34.318 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:34.318 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:10:34.318 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:34.318 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:34.319 INFO:teuthology.orchestra.run.smithi022.stdout:Dependency Installed: 2019-12-09T09:10:34.319 INFO:teuthology.orchestra.run.smithi022.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-09T09:10:34.319 INFO:teuthology.orchestra.run.smithi022.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-09T09:10:34.319 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:34.319 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:10:34.434 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph 2019-12-09T09:10:34.699 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:10:34.745 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:10:34.745 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:10:34.746 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:10:34.746 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:10:34.746 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:10:35.137 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-12-09T09:10:35.409 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:10:35.667 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-12-09T09:10:35.764 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:10:35.765 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:35.765 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:35.765 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:35.793 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:35.794 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:35.795 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:35.797 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:35.797 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:35.809 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:35.809 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:35.811 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:35.834 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:35.835 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:35.835 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:35.835 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:10:35.835 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:35.835 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:35.835 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:35.836 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-09T09:10:35.836 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-09T09:10:35.836 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-09T09:10:35.837 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:10:35.837 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:10:35.837 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:10:35.838 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:10:35.838 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:10:35.838 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-12-09T09:10:35.838 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-12-09T09:10:35.839 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-09T09:10:35.839 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-12-09T09:10:35.839 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-12-09T09:10:35.839 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:35.839 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-09T09:10:35.840 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:10:35.840 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:10:35.843 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-09T09:10:35.844 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-09T09:10:35.844 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-09T09:10:35.844 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-09T09:10:35.844 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-09T09:10:35.845 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-12-09T09:10:35.845 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:35.845 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-09T09:10:35.845 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-09T09:10:35.845 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-09T09:10:35.848 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:35.848 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-09T09:10:35.849 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-09T09:10:35.849 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:35.849 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-09T09:10:36.010 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-12-09T09:10:36.217 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:10:36.451 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:36.451 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:10:36.452 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:36.452 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:36.452 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository 2019-12-09T09:10:36.452 INFO:teuthology.orchestra.run.smithi022.stdout: Size 2019-12-09T09:10:36.452 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:36.452 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:10:36.453 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.0 k 2019-12-09T09:10:36.453 INFO:teuthology.orchestra.run.smithi022.stdout:Installing for dependencies: 2019-12-09T09:10:36.454 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.9 M 2019-12-09T09:10:36.454 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.7 M 2019-12-09T09:10:36.454 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 4.4 M 2019-12-09T09:10:36.454 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 16 M 2019-12-09T09:10:36.454 INFO:teuthology.orchestra.run.smithi022.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-09T09:10:36.455 INFO:teuthology.orchestra.run.smithi022.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-09T09:10:36.455 INFO:teuthology.orchestra.run.smithi022.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-09T09:10:36.455 INFO:teuthology.orchestra.run.smithi022.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-09T09:10:36.455 INFO:teuthology.orchestra.run.smithi022.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-09T09:10:36.455 INFO:teuthology.orchestra.run.smithi022.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-09T09:10:36.455 INFO:teuthology.orchestra.run.smithi022.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-09T09:10:36.455 INFO:teuthology.orchestra.run.smithi022.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-12-09T09:10:36.456 INFO:teuthology.orchestra.run.smithi022.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-09T09:10:36.456 INFO:teuthology.orchestra.run.smithi022.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-09T09:10:36.456 INFO:teuthology.orchestra.run.smithi022.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-09T09:10:36.456 INFO:teuthology.orchestra.run.smithi022.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-09T09:10:36.456 INFO:teuthology.orchestra.run.smithi022.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-09T09:10:36.456 INFO:teuthology.orchestra.run.smithi022.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-09T09:10:36.456 INFO:teuthology.orchestra.run.smithi022.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-09T09:10:36.456 INFO:teuthology.orchestra.run.smithi022.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-12-09T09:10:36.457 INFO:teuthology.orchestra.run.smithi022.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-12-09T09:10:36.457 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:36.457 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:10:36.457 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:36.457 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package (+21 Dependent packages) 2019-12-09T09:10:36.457 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:36.458 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 28 M 2019-12-09T09:10:36.458 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 87 M 2019-12-09T09:10:36.459 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:10:36.583 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-12-09T09:10:37.103 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-12-09T09:10:37.951 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:10:37.951 INFO:teuthology.orchestra.run.smithi022.stdout:Total 18 MB/s | 28 MB 00:01 2019-12-09T09:10:37.981 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:10:38.026 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:10:38.282 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:10:38.283 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:10:39.045 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-12-09T09:10:39.198 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-12-09T09:10:39.340 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-12-09T09:10:39.571 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-12-09T09:10:39.657 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 9/22 2019-12-09T09:10:40.034 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-12-09T09:10:40.064 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 10/22 2019-12-09T09:10:40.375 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-mako-0.8.1-2.el7.noarch 11/22 2019-12-09T09:10:40.391 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-12-09T09:10:40.822 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-12-09T09:10:41.314 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-12-09T09:10:41.368 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-09T09:10:41.829 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:10:42.189 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-12-09T09:10:42.896 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-12-09T09:10:43.214 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-12-09T09:10:43.417 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 16/22 2019-12-09T09:10:43.591 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 9/22 2019-12-09T09:10:43.672 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-12-09T09:10:43.941 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 10/22 2019-12-09T09:10:44.018 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-12-09T09:10:44.252 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-mako-0.8.1-2.el7.noarch 11/22 2019-12-09T09:10:44.630 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-12-09T09:10:45.021 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-09T09:10:45.319 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-12-09T09:10:45.475 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:10:45.774 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 21/22 2019-12-09T09:10:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-12-09T09:10:46.077 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 22/22 2019-12-09T09:10:46.223 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-12-09T09:10:46.320 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-12-09T09:10:46.355 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-12-09T09:10:46.499 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-12-09T09:10:46.624 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-12-09T09:10:46.659 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 16/22 2019-12-09T09:10:46.750 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 5/22 2019-12-09T09:10:46.867 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-09T09:10:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-12-09T09:10:47.000 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 7/22 2019-12-09T09:10:47.093 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 8/22 2019-12-09T09:10:47.170 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-12-09T09:10:47.202 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 9/22 2019-12-09T09:10:47.321 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 10/22 2019-12-09T09:10:47.403 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-12-09T09:10:47.522 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-12-09T09:10:47.653 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:10:47.678 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-12-09T09:10:47.771 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 14/22 2019-12-09T09:10:47.929 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-12-09T09:10:48.038 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-12-09T09:10:48.205 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-12-09T09:10:48.345 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 18/22 2019-12-09T09:10:48.498 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 19/22 2019-12-09T09:10:48.539 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-12-09T09:10:48.607 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 20/22 2019-12-09T09:10:48.707 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 21/22 2019-12-09T09:10:48.925 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 21/22 2019-12-09T09:10:49.188 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 22/22 2019-12-09T09:10:49.228 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-09T09:10:49.229 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:49.229 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:10:49.229 INFO:teuthology.orchestra.run.smithi035.stdout: ceph.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:49.229 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:49.229 INFO:teuthology.orchestra.run.smithi035.stdout:Dependency Installed: 2019-12-09T09:10:49.229 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:49.230 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:49.230 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:49.230 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:49.230 INFO:teuthology.orchestra.run.smithi035.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-09T09:10:49.230 INFO:teuthology.orchestra.run.smithi035.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-09T09:10:49.230 INFO:teuthology.orchestra.run.smithi035.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-09T09:10:49.230 INFO:teuthology.orchestra.run.smithi035.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-09T09:10:49.231 INFO:teuthology.orchestra.run.smithi035.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-09T09:10:49.231 INFO:teuthology.orchestra.run.smithi035.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-09T09:10:49.231 INFO:teuthology.orchestra.run.smithi035.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-09T09:10:49.231 INFO:teuthology.orchestra.run.smithi035.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-12-09T09:10:49.231 INFO:teuthology.orchestra.run.smithi035.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-09T09:10:49.231 INFO:teuthology.orchestra.run.smithi035.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-09T09:10:49.232 INFO:teuthology.orchestra.run.smithi035.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-09T09:10:49.232 INFO:teuthology.orchestra.run.smithi035.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-09T09:10:49.232 INFO:teuthology.orchestra.run.smithi035.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-09T09:10:49.232 INFO:teuthology.orchestra.run.smithi035.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-09T09:10:49.232 INFO:teuthology.orchestra.run.smithi035.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-09T09:10:49.233 INFO:teuthology.orchestra.run.smithi035.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-12-09T09:10:49.233 INFO:teuthology.orchestra.run.smithi035.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-12-09T09:10:49.233 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:49.233 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:10:49.291 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-12-09T09:10:49.401 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-12-09T09:10:49.402 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-daemon 2019-12-09T09:10:49.525 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-12-09T09:10:49.626 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-12-09T09:10:49.693 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:10:49.728 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 5/22 2019-12-09T09:10:49.739 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:10:49.740 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:10:49.740 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:10:49.741 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:10:49.741 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:10:49.837 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-09T09:10:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 7/22 2019-12-09T09:10:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 8/22 2019-12-09T09:10:50.122 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 9/22 2019-12-09T09:10:50.214 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 10/22 2019-12-09T09:10:50.315 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-12-09T09:10:50.421 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:10:50.423 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-12-09T09:10:50.523 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:10:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 14/22 2019-12-09T09:10:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-12-09T09:10:50.772 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:10:50.773 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:50.773 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-12-09T09:10:50.960 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-12-09T09:10:51.076 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 18/22 2019-12-09T09:10:51.158 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:10:51.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 19/22 2019-12-09T09:10:51.318 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 20/22 2019-12-09T09:10:51.387 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:51.387 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:10:51.388 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:51.388 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:51.388 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository 2019-12-09T09:10:51.388 INFO:teuthology.orchestra.run.smithi035.stdout: Size 2019-12-09T09:10:51.388 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:51.389 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:10:51.389 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-daemon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:10:51.389 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:51.389 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:10:51.389 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:51.389 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2019-12-09T09:10:51.389 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:51.389 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 17 k 2019-12-09T09:10:51.390 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 68 k 2019-12-09T09:10:51.390 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:10:51.419 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 21/22 2019-12-09T09:10:51.672 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:10:51.677 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:10:51.679 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:10:51.679 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:10:51.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-09T09:10:51.930 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:51.930 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:10:51.930 INFO:teuthology.orchestra.run.smithi022.stdout: ceph.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:51.930 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:51.931 INFO:teuthology.orchestra.run.smithi022.stdout:Dependency Installed: 2019-12-09T09:10:51.931 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:51.931 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:51.931 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:51.932 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:51.932 INFO:teuthology.orchestra.run.smithi022.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-09T09:10:51.932 INFO:teuthology.orchestra.run.smithi022.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-09T09:10:51.933 INFO:teuthology.orchestra.run.smithi022.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-09T09:10:51.933 INFO:teuthology.orchestra.run.smithi022.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-09T09:10:51.933 INFO:teuthology.orchestra.run.smithi022.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-09T09:10:51.933 INFO:teuthology.orchestra.run.smithi022.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-09T09:10:51.933 INFO:teuthology.orchestra.run.smithi022.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-09T09:10:51.934 INFO:teuthology.orchestra.run.smithi022.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-12-09T09:10:51.934 INFO:teuthology.orchestra.run.smithi022.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-09T09:10:51.934 INFO:teuthology.orchestra.run.smithi022.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-09T09:10:51.934 INFO:teuthology.orchestra.run.smithi022.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-09T09:10:51.934 INFO:teuthology.orchestra.run.smithi022.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-09T09:10:51.935 INFO:teuthology.orchestra.run.smithi022.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-09T09:10:51.935 INFO:teuthology.orchestra.run.smithi022.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-09T09:10:51.935 INFO:teuthology.orchestra.run.smithi022.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-09T09:10:51.935 INFO:teuthology.orchestra.run.smithi022.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-12-09T09:10:51.935 INFO:teuthology.orchestra.run.smithi022.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-12-09T09:10:51.935 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:51.936 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:10:52.063 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-daemon 2019-12-09T09:10:52.309 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:10:52.344 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:10:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:10:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:10:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:10:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:10:52.588 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:10:53.029 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:10:53.043 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:10:53.043 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:53.044 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:10:53.044 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:53.044 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:53.044 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:10:53.167 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-mgr 2019-12-09T09:10:53.384 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:10:53.385 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:53.385 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:53.442 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:10:53.487 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:10:53.488 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:10:53.489 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:10:53.489 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:10:53.489 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:10:53.770 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:10:54.002 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:54.002 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:10:54.003 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:54.003 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:54.003 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository 2019-12-09T09:10:54.003 INFO:teuthology.orchestra.run.smithi022.stdout: Size 2019-12-09T09:10:54.004 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:54.004 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:10:54.004 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-daemon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:10:54.005 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:54.006 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:10:54.007 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:54.009 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2019-12-09T09:10:54.009 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:54.009 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 17 k 2019-12-09T09:10:54.011 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 68 k 2019-12-09T09:10:54.011 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:10:54.169 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:10:54.300 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:10:54.309 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:10:54.310 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:10:54.311 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:10:54.520 INFO:teuthology.orchestra.run.smithi035.stdout:Package 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:10:54.521 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do 2019-12-09T09:10:54.582 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-mgr-dashboard 2019-12-09T09:10:54.752 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:10:54.793 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:10:54.793 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:10:54.793 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:10:54.793 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:10:54.793 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:10:55.072 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:10:55.430 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:10:55.526 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:10:55.526 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:55.526 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:10:55.526 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:10:55.526 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:55.527 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:10:55.671 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr 2019-12-09T09:10:55.773 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:10:55.774 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:55.774 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:55.776 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:10:55.828 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:10:55.828 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:55.828 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:55.828 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-12-09T09:10:55.828 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-12-09T09:10:55.829 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:10:55.829 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-12-09T09:10:55.939 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:10:55.984 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:10:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:10:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:10:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:10:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:10:56.198 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:10:56.430 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:56.431 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:10:56.432 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:56.432 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:56.432 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2019-12-09T09:10:56.432 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:56.432 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:10:56.432 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 3.9 M 2019-12-09T09:10:56.433 INFO:teuthology.orchestra.run.smithi035.stdout:Installing for dependencies: 2019-12-09T09:10:56.433 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 18 k 2019-12-09T09:10:56.433 INFO:teuthology.orchestra.run.smithi035.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-12-09T09:10:56.433 INFO:teuthology.orchestra.run.smithi035.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-12-09T09:10:56.433 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:56.433 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:10:56.433 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:10:56.434 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package (+3 Dependent packages) 2019-12-09T09:10:56.434 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:10:56.434 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 4.6 M 2019-12-09T09:10:56.434 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 20 M 2019-12-09T09:10:56.434 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:10:56.667 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:10:57.021 INFO:teuthology.orchestra.run.smithi022.stdout:Package 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:10:57.021 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do 2019-12-09T09:10:57.051 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:10:57.051 INFO:teuthology.orchestra.run.smithi035.stdout:Total 7.4 MB/s | 4.6 MB 00:00 2019-12-09T09:10:57.057 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:10:57.065 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:10:57.085 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-dashboard 2019-12-09T09:10:57.254 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:10:57.255 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:10:57.278 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:10:57.317 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:10:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:10:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:10:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:10:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:10:57.925 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 1/4 2019-12-09T09:10:57.977 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:10:58.318 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:10:58.318 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:58.319 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:58.322 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:10:58.353 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:10:58.354 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:58.354 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:10:58.354 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-12-09T09:10:58.354 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-12-09T09:10:58.355 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:10:58.355 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-12-09T09:10:58.392 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-12-09T09:10:58.740 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:10:58.972 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:58.972 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:10:58.973 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:58.973 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:58.974 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2019-12-09T09:10:58.974 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:58.974 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:10:58.974 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 3.9 M 2019-12-09T09:10:58.974 INFO:teuthology.orchestra.run.smithi022.stdout:Installing for dependencies: 2019-12-09T09:10:58.974 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 18 k 2019-12-09T09:10:58.974 INFO:teuthology.orchestra.run.smithi022.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-12-09T09:10:58.975 INFO:teuthology.orchestra.run.smithi022.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-12-09T09:10:58.975 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:58.975 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:10:58.975 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:10:58.975 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package (+3 Dependent packages) 2019-12-09T09:10:58.975 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:10:58.976 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 4.6 M 2019-12-09T09:10:58.976 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 20 M 2019-12-09T09:10:58.976 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:10:59.582 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-12-09T09:10:59.714 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:10:59.715 INFO:teuthology.orchestra.run.smithi022.stdout:Total 6.2 MB/s | 4.6 MB 00:00 2019-12-09T09:10:59.721 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:10:59.732 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:10:59.884 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:10:59.885 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:11:00.513 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 1/4 2019-12-09T09:11:00.730 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 4/4 2019-12-09T09:11:00.890 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-12-09T09:11:00.905 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-12-09T09:11:01.024 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2/4 2019-12-09T09:11:01.149 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 3/4 2019-12-09T09:11:01.576 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-12-09T09:11:01.577 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:01.577 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:11:01.577 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:01.577 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:01.577 INFO:teuthology.orchestra.run.smithi035.stdout:Dependency Installed: 2019-12-09T09:11:01.578 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:01.578 INFO:teuthology.orchestra.run.smithi035.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-12-09T09:11:01.578 INFO:teuthology.orchestra.run.smithi035.stdout: python-routes.noarch 0:1.13-2.el7 2019-12-09T09:11:01.578 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:01.579 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:11:01.706 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-12-09T09:11:01.942 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:01.982 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:01.983 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:01.984 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:01.984 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:01.984 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:02.054 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-12-09T09:11:02.430 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 4/4 2019-12-09T09:11:02.529 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-12-09T09:11:02.674 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:03.022 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:11:03.023 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:03.023 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:03.409 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:03.640 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:03.640 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:11:03.641 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:03.641 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:03.641 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2019-12-09T09:11:03.641 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:03.641 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:11:03.641 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-diskprediction-cloud 2019-12-09T09:11:03.642 INFO:teuthology.orchestra.run.smithi035.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 82 k 2019-12-09T09:11:03.642 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:03.642 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:11:03.642 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:03.642 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2019-12-09T09:11:03.642 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:03.642 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 82 k 2019-12-09T09:11:03.642 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 411 k 2019-12-09T09:11:03.643 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:11:03.689 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2/4 2019-12-09T09:11:03.789 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 3/4 2019-12-09T09:11:03.974 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:11:03.978 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:11:04.031 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:11:04.032 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:11:04.283 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-12-09T09:11:04.284 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:04.284 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:11:04.284 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:04.284 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:04.284 INFO:teuthology.orchestra.run.smithi022.stdout:Dependency Installed: 2019-12-09T09:11:04.285 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:04.285 INFO:teuthology.orchestra.run.smithi022.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-12-09T09:11:04.285 INFO:teuthology.orchestra.run.smithi022.stdout: python-routes.noarch 0:1.13-2.el7 2019-12-09T09:11:04.285 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:04.285 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:11:04.405 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-12-09T09:11:04.648 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:04.694 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:04.695 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:04.695 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:04.695 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:04.696 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:04.724 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:11:05.213 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:11:05.213 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:05.213 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:11:05.213 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:05.214 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:05.214 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:11:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:05.414 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-mgr-diskprediction-local 2019-12-09T09:11:05.676 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:05.722 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:05.723 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:05.724 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:05.724 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:05.724 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:05.729 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:11:05.730 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:05.730 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:06.117 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:06.347 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:06.347 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:11:06.348 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:06.348 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:06.348 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2019-12-09T09:11:06.349 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:06.349 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:11:06.349 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-cloud 2019-12-09T09:11:06.349 INFO:teuthology.orchestra.run.smithi022.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 82 k 2019-12-09T09:11:06.349 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:06.349 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:11:06.349 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:06.350 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2019-12-09T09:11:06.350 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:06.350 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 82 k 2019-12-09T09:11:06.350 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 411 k 2019-12-09T09:11:06.350 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:11:06.397 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:06.677 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:11:06.680 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:11:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:11:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:11:06.748 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:11:06.748 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:06.748 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:06.750 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:11:06.782 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:06.782 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-12-09T09:11:06.789 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:11:06.790 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:11:06.793 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:11:06.793 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:06.793 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-12-09T09:11:06.794 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-12-09T09:11:06.795 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:11:06.797 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:11:06.797 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:06.797 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-12-09T09:11:07.182 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:07.371 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:11:07.415 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:07.415 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:11:07.416 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:07.416 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:07.416 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2019-12-09T09:11:07.416 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:07.416 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:11:07.416 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-diskprediction-local 2019-12-09T09:11:07.417 INFO:teuthology.orchestra.run.smithi035.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 8.6 M 2019-12-09T09:11:07.417 INFO:teuthology.orchestra.run.smithi035.stdout:Installing for dependencies: 2019-12-09T09:11:07.417 INFO:teuthology.orchestra.run.smithi035.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-12-09T09:11:07.417 INFO:teuthology.orchestra.run.smithi035.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-12-09T09:11:07.417 INFO:teuthology.orchestra.run.smithi035.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-12-09T09:11:07.417 INFO:teuthology.orchestra.run.smithi035.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-12-09T09:11:07.417 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:07.418 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:11:07.418 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:07.418 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:11:07.418 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:07.418 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 19 M 2019-12-09T09:11:07.418 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 104 M 2019-12-09T09:11:07.418 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:11:07.879 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:11:07.879 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:07.879 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:11:07.879 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:07.879 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:07.880 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:11:08.012 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-diskprediction-local 2019-12-09T09:11:08.242 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:08.285 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:08.287 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:08.790 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:11:08.791 INFO:teuthology.orchestra.run.smithi035.stdout:Total 14 MB/s | 19 MB 00:01 2019-12-09T09:11:08.805 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:11:08.834 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:11:08.950 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:09.008 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:11:09.008 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:11:09.299 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:11:09.299 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:09.299 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:09.301 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:11:09.332 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:09.333 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-12-09T09:11:09.340 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:11:09.340 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:11:09.343 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:11:09.343 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:09.344 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-12-09T09:11:09.344 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-12-09T09:11:09.345 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:11:09.347 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:11:09.347 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:09.347 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-12-09T09:11:09.728 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:09.955 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-12-09T09:11:09.957 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:09.957 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:11:09.958 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:09.958 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:09.959 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2019-12-09T09:11:09.959 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:09.959 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:11:09.959 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local 2019-12-09T09:11:09.959 INFO:teuthology.orchestra.run.smithi022.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 8.6 M 2019-12-09T09:11:09.959 INFO:teuthology.orchestra.run.smithi022.stdout:Installing for dependencies: 2019-12-09T09:11:09.959 INFO:teuthology.orchestra.run.smithi022.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-12-09T09:11:09.960 INFO:teuthology.orchestra.run.smithi022.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-12-09T09:11:09.960 INFO:teuthology.orchestra.run.smithi022.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-12-09T09:11:09.960 INFO:teuthology.orchestra.run.smithi022.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-12-09T09:11:09.960 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:09.960 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:11:09.960 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:09.960 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:11:09.961 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:09.961 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 19 M 2019-12-09T09:11:09.961 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 104 M 2019-12-09T09:11:09.961 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:11:10.778 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:11:10.779 INFO:teuthology.orchestra.run.smithi022.stdout:Total 23 MB/s | 19 MB 00:00 2019-12-09T09:11:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:11:10.825 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:11:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:11:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:11:10.996 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-12-09T09:11:11.823 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-12-09T09:11:12.289 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-12-09T09:11:12.795 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-12-09T09:11:14.072 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-12-09T09:11:14.503 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-12-09T09:11:14.842 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 5/5 2019-12-09T09:11:14.987 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-12-09T09:11:15.146 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 2/5 2019-12-09T09:11:15.731 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 3/5 2019-12-09T09:11:15.763 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-12-09T09:11:15.823 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-12-09T09:11:16.061 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 5/5 2019-12-09T09:11:16.182 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-12-09T09:11:16.258 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-12-09T09:11:16.259 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:16.259 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:11:16.259 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:16.259 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:16.259 INFO:teuthology.orchestra.run.smithi035.stdout:Dependency Installed: 2019-12-09T09:11:16.260 INFO:teuthology.orchestra.run.smithi035.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-12-09T09:11:16.260 INFO:teuthology.orchestra.run.smithi035.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-12-09T09:11:16.260 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:16.260 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:11:16.291 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 2/5 2019-12-09T09:11:16.396 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-mgr-rook 2019-12-09T09:11:16.400 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 3/5 2019-12-09T09:11:16.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-12-09T09:11:16.668 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:16.715 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:16.716 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:16.717 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:16.717 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:16.717 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:16.952 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-12-09T09:11:16.952 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:16.953 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:11:16.953 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:16.953 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:16.953 INFO:teuthology.orchestra.run.smithi022.stdout:Dependency Installed: 2019-12-09T09:11:16.953 INFO:teuthology.orchestra.run.smithi022.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-12-09T09:11:16.953 INFO:teuthology.orchestra.run.smithi022.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-12-09T09:11:16.953 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:16.954 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:11:17.087 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-rook 2019-12-09T09:11:17.322 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:17.363 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:17.364 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:17.365 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:17.365 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:17.365 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:17.402 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:17.758 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:11:17.758 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:17.758 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:17.760 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:11:17.791 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:17.792 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-12-09T09:11:17.794 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:11:17.795 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:11:17.796 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:11:17.797 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:11:17.798 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:17.798 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-12-09T09:11:17.799 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-12-09T09:11:17.799 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-12-09T09:11:17.799 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:11:17.800 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:11:17.801 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:11:17.802 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-12-09T09:11:17.802 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-12-09T09:11:17.803 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:17.804 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-12-09T09:11:17.804 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-12-09T09:11:17.804 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-12-09T09:11:17.806 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-12-09T09:11:17.806 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-12-09T09:11:17.806 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:17.806 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-12-09T09:11:18.043 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:18.189 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:18.388 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:11:18.388 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:18.388 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:18.390 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:11:18.419 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:18.420 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:11:18.422 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:18.422 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-12-09T09:11:18.423 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:18.423 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:18.423 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2019-12-09T09:11:18.423 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:18.423 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:11:18.424 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-rook noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 23 k 2019-12-09T09:11:18.424 INFO:teuthology.orchestra.run.smithi035.stdout:Installing for dependencies: 2019-12-09T09:11:18.424 INFO:teuthology.orchestra.run.smithi035.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-12-09T09:11:18.424 INFO:teuthology.orchestra.run.smithi035.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-12-09T09:11:18.424 INFO:teuthology.orchestra.run.smithi035.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-12-09T09:11:18.424 INFO:teuthology.orchestra.run.smithi035.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-12-09T09:11:18.425 INFO:teuthology.orchestra.run.smithi035.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-12-09T09:11:18.425 INFO:teuthology.orchestra.run.smithi035.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-12-09T09:11:18.425 INFO:teuthology.orchestra.run.smithi035.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-12-09T09:11:18.425 INFO:teuthology.orchestra.run.smithi035.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-12-09T09:11:18.425 INFO:teuthology.orchestra.run.smithi035.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-12-09T09:11:18.425 INFO:teuthology.orchestra.run.smithi035.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-12-09T09:11:18.425 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:18.426 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:11:18.426 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:18.426 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package (+10 Dependent packages) 2019-12-09T09:11:18.426 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:18.426 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 1.6 M 2019-12-09T09:11:18.427 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 24 M 2019-12-09T09:11:18.427 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:11:18.429 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:11:18.429 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:11:18.429 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:11:18.429 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:11:18.429 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:18.429 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-12-09T09:11:18.429 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-12-09T09:11:18.430 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-12-09T09:11:18.430 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:11:18.431 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:11:18.432 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:11:18.433 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-12-09T09:11:18.433 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-12-09T09:11:18.434 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:18.434 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-12-09T09:11:18.435 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-12-09T09:11:18.436 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-12-09T09:11:18.436 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-12-09T09:11:18.436 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-12-09T09:11:18.436 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:18.437 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-12-09T09:11:18.820 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:19.058 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:11:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:19.059 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2019-12-09T09:11:19.059 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:19.059 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:11:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 23 k 2019-12-09T09:11:19.059 INFO:teuthology.orchestra.run.smithi022.stdout:Installing for dependencies: 2019-12-09T09:11:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-12-09T09:11:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-12-09T09:11:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-12-09T09:11:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-12-09T09:11:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-12-09T09:11:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-12-09T09:11:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-12-09T09:11:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-12-09T09:11:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-12-09T09:11:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-12-09T09:11:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:19.062 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:11:19.062 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:19.063 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package (+10 Dependent packages) 2019-12-09T09:11:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:19.063 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 1.6 M 2019-12-09T09:11:19.063 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 24 M 2019-12-09T09:11:19.068 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:11:19.381 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:11:19.381 INFO:teuthology.orchestra.run.smithi035.stdout:Total 1.7 MB/s | 1.6 MB 00:00 2019-12-09T09:11:19.395 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:11:19.409 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:11:19.484 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:11:19.485 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:11:19.950 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:11:19.951 INFO:teuthology.orchestra.run.smithi022.stdout:Total 1.8 MB/s | 1.6 MB 00:00 2019-12-09T09:11:19.968 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:11:19.982 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:11:20.058 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:11:20.059 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:11:20.266 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-12-09T09:11:20.696 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-12-09T09:11:20.785 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-12-09T09:11:21.044 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-12-09T09:11:21.157 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-12-09T09:11:21.392 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-12-09T09:11:21.496 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-12-09T09:11:21.742 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-12-09T09:11:21.771 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-12-09T09:11:22.116 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-12-09T09:11:22.143 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-12-09T09:11:22.438 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-12-09T09:11:22.529 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-12-09T09:11:22.790 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-12-09T09:11:22.899 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-12-09T09:11:23.114 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-12-09T09:11:24.206 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-12-09T09:11:24.467 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-12-09T09:11:24.617 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-12-09T09:11:24.868 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-12-09T09:11:24.928 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 11/11 2019-12-09T09:11:25.094 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-12-09T09:11:25.138 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 11/11 2019-12-09T09:11:25.243 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-12-09T09:11:25.253 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-12-09T09:11:25.378 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-12-09T09:11:25.386 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-12-09T09:11:25.495 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-12-09T09:11:25.529 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-12-09T09:11:25.613 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-12-09T09:11:25.620 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-12-09T09:11:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-12-09T09:11:25.721 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-12-09T09:11:25.814 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-12-09T09:11:25.830 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-12-09T09:11:25.947 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-12-09T09:11:25.948 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-12-09T09:11:26.040 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-12-09T09:11:26.064 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-12-09T09:11:26.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-12-09T09:11:26.148 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 10/11 2019-12-09T09:11:26.224 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 10/11 2019-12-09T09:11:26.667 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 11/11 2019-12-09T09:11:26.668 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:26.668 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:11:26.668 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:26.668 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:26.668 INFO:teuthology.orchestra.run.smithi022.stdout:Dependency Installed: 2019-12-09T09:11:26.668 INFO:teuthology.orchestra.run.smithi022.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-12-09T09:11:26.669 INFO:teuthology.orchestra.run.smithi022.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-12-09T09:11:26.669 INFO:teuthology.orchestra.run.smithi022.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-12-09T09:11:26.669 INFO:teuthology.orchestra.run.smithi022.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-12-09T09:11:26.670 INFO:teuthology.orchestra.run.smithi022.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-12-09T09:11:26.670 INFO:teuthology.orchestra.run.smithi022.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-12-09T09:11:26.670 INFO:teuthology.orchestra.run.smithi022.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-12-09T09:11:26.670 INFO:teuthology.orchestra.run.smithi022.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-12-09T09:11:26.670 INFO:teuthology.orchestra.run.smithi022.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-12-09T09:11:26.670 INFO:teuthology.orchestra.run.smithi022.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-12-09T09:11:26.671 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:26.671 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:11:26.709 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 11/11 2019-12-09T09:11:26.709 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:26.709 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:11:26.709 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:26.710 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:26.710 INFO:teuthology.orchestra.run.smithi035.stdout:Dependency Installed: 2019-12-09T09:11:26.710 INFO:teuthology.orchestra.run.smithi035.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-12-09T09:11:26.710 INFO:teuthology.orchestra.run.smithi035.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-12-09T09:11:26.710 INFO:teuthology.orchestra.run.smithi035.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-12-09T09:11:26.710 INFO:teuthology.orchestra.run.smithi035.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-12-09T09:11:26.711 INFO:teuthology.orchestra.run.smithi035.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-12-09T09:11:26.711 INFO:teuthology.orchestra.run.smithi035.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-12-09T09:11:26.711 INFO:teuthology.orchestra.run.smithi035.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-12-09T09:11:26.711 INFO:teuthology.orchestra.run.smithi035.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-12-09T09:11:26.711 INFO:teuthology.orchestra.run.smithi035.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-12-09T09:11:26.711 INFO:teuthology.orchestra.run.smithi035.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-12-09T09:11:26.711 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:26.711 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:11:26.787 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-ssh 2019-12-09T09:11:26.832 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-mgr-ssh 2019-12-09T09:11:27.031 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:27.051 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:27.076 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:27.076 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:27.076 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:27.076 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:27.077 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:27.090 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:27.091 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:27.092 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:27.092 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:27.092 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:27.745 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:27.752 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:28.095 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:11:28.095 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:28.095 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:28.095 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:11:28.095 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:28.096 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:28.097 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:11:28.097 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:11:28.128 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:28.129 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-12-09T09:11:28.131 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:28.131 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-12-09T09:11:28.131 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-12-09T09:11:28.131 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:28.131 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-12-09T09:11:28.132 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-12-09T09:11:28.132 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:28.132 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-12-09T09:11:28.519 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:28.520 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:28.754 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:28.754 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:11:28.755 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:28.756 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:11:28.756 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:28.756 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:28.756 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2019-12-09T09:11:28.756 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:28.756 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:11:28.756 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-ssh noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 33 k 2019-12-09T09:11:28.757 INFO:teuthology.orchestra.run.smithi035.stdout:Installing for dependencies: 2019-12-09T09:11:28.757 INFO:teuthology.orchestra.run.smithi035.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-12-09T09:11:28.758 INFO:teuthology.orchestra.run.smithi035.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-12-09T09:11:28.758 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:28.758 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:11:28.758 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:28.758 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package (+2 Dependent packages) 2019-12-09T09:11:28.758 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:28.758 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 316 k 2019-12-09T09:11:28.759 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 1.1 M 2019-12-09T09:11:28.759 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:11:28.759 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:28.759 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:28.759 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2019-12-09T09:11:28.759 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:28.759 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:11:28.760 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-ssh noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 33 k 2019-12-09T09:11:28.760 INFO:teuthology.orchestra.run.smithi022.stdout:Installing for dependencies: 2019-12-09T09:11:28.760 INFO:teuthology.orchestra.run.smithi022.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-12-09T09:11:28.760 INFO:teuthology.orchestra.run.smithi022.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-12-09T09:11:28.760 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:28.760 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:11:28.760 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:28.761 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package (+2 Dependent packages) 2019-12-09T09:11:28.761 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:28.761 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 316 k 2019-12-09T09:11:28.761 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 1.1 M 2019-12-09T09:11:28.761 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:11:29.290 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:11:29.291 INFO:teuthology.orchestra.run.smithi022.stdout:Total 591 kB/s | 316 kB 00:00 2019-12-09T09:11:29.294 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:11:29.296 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:11:29.296 INFO:teuthology.orchestra.run.smithi035.stdout:Total 585 kB/s | 316 kB 00:00 2019-12-09T09:11:29.300 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:11:29.303 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:11:29.308 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:11:29.379 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:11:29.380 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:11:29.395 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:11:29.396 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:11:30.156 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-12-09T09:11:30.247 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-12-09T09:11:30.416 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-12-09T09:11:30.693 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:11:30.834 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-12-09T09:11:30.955 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-12-09T09:11:30.978 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-12-09T09:11:31.303 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:11:31.365 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:11:31.365 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:31.365 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:11:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:31.366 INFO:teuthology.orchestra.run.smithi022.stdout:Dependency Installed: 2019-12-09T09:11:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-12-09T09:11:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:31.366 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:11:31.460 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-12-09T09:11:31.495 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-fuse 2019-12-09T09:11:31.601 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-12-09T09:11:31.757 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:31.807 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:32.111 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:11:32.111 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:32.111 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:11:32.111 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:32.111 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:32.112 INFO:teuthology.orchestra.run.smithi035.stdout:Dependency Installed: 2019-12-09T09:11:32.112 INFO:teuthology.orchestra.run.smithi035.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-12-09T09:11:32.112 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:32.112 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:11:32.229 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-fuse 2019-12-09T09:11:32.478 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:32.503 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:32.550 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:32.550 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:32.551 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:32.551 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:32.552 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:32.836 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:11:32.837 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:32.837 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:32.850 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:11:32.882 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:11:32.883 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:32.883 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:11:32.883 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:11:32.889 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:11:32.889 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:11:32.889 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:11:32.890 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:32.890 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:11:32.893 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-12-09T09:11:32.894 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-12-09T09:11:33.220 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:33.281 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:33.513 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:33.513 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:11:33.514 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:33.515 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:33.515 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository 2019-12-09T09:11:33.515 INFO:teuthology.orchestra.run.smithi022.stdout: Size 2019-12-09T09:11:33.515 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:33.516 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:11:33.516 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 566 k 2019-12-09T09:11:33.516 INFO:teuthology.orchestra.run.smithi022.stdout:Installing for dependencies: 2019-12-09T09:11:33.516 INFO:teuthology.orchestra.run.smithi022.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-12-09T09:11:33.518 INFO:teuthology.orchestra.run.smithi022.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-12-09T09:11:33.518 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-12-09T09:11:33.518 INFO:teuthology.orchestra.run.smithi022.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-12-09T09:11:33.519 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:33.519 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:11:33.519 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:33.519 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:11:33.520 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:33.520 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 10 M 2019-12-09T09:11:33.520 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 49 M 2019-12-09T09:11:33.520 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:11:33.575 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:11:33.576 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:33.576 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:33.587 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:11:33.626 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:11:33.626 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:33.626 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:11:33.627 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:11:33.627 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:11:33.627 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:11:33.627 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:11:33.629 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:33.629 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:11:33.636 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-12-09T09:11:33.636 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-12-09T09:11:33.978 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:11:33.978 INFO:teuthology.orchestra.run.smithi022.stdout:Total 22 MB/s | 10 MB 00:00 2019-12-09T09:11:34.000 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:11:34.016 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:34.029 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:11:34.169 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:11:34.170 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:11:34.252 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:34.252 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:11:34.253 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:34.253 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:34.253 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository 2019-12-09T09:11:34.254 INFO:teuthology.orchestra.run.smithi035.stdout: Size 2019-12-09T09:11:34.254 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:34.254 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:11:34.254 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 566 k 2019-12-09T09:11:34.254 INFO:teuthology.orchestra.run.smithi035.stdout:Installing for dependencies: 2019-12-09T09:11:34.255 INFO:teuthology.orchestra.run.smithi035.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-12-09T09:11:34.255 INFO:teuthology.orchestra.run.smithi035.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-12-09T09:11:34.255 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-12-09T09:11:34.255 INFO:teuthology.orchestra.run.smithi035.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-12-09T09:11:34.255 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:34.255 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:11:34.256 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:34.256 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:11:34.256 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:34.256 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 10 M 2019-12-09T09:11:34.256 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 49 M 2019-12-09T09:11:34.256 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:11:34.627 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:11:34.628 INFO:teuthology.orchestra.run.smithi035.stdout:Total 27 MB/s | 10 MB 00:00 2019-12-09T09:11:34.641 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:11:34.662 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:11:34.802 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:11:34.802 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:11:36.490 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-12-09T09:11:36.992 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-12-09T09:11:37.126 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-12-09T09:11:37.653 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-12-09T09:11:37.885 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-12-09T09:11:38.335 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-12-09T09:11:38.567 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-12-09T09:11:38.593 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:11:38.709 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:11:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-12-09T09:11:38.944 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-12-09T09:11:39.019 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-12-09T09:11:39.060 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-12-09T09:11:39.279 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:11:39.370 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:11:39.504 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-12-09T09:11:39.537 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-12-09T09:11:39.537 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:39.537 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:11:39.537 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:39.537 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:39.537 INFO:teuthology.orchestra.run.smithi022.stdout:Dependency Installed: 2019-12-09T09:11:39.538 INFO:teuthology.orchestra.run.smithi022.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-12-09T09:11:39.538 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-12-09T09:11:39.538 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:39.538 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:11:39.596 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-12-09T09:11:39.656 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install libcephfs2 2019-12-09T09:11:39.739 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-12-09T09:11:39.927 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:39.971 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:39.972 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:39.972 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:39.972 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:39.975 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:40.257 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-12-09T09:11:40.257 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:40.257 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:11:40.258 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:40.258 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:40.258 INFO:teuthology.orchestra.run.smithi035.stdout:Dependency Installed: 2019-12-09T09:11:40.258 INFO:teuthology.orchestra.run.smithi035.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-12-09T09:11:40.259 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-12-09T09:11:40.259 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:40.259 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:11:40.382 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install libcephfs2 2019-12-09T09:11:40.639 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:40.652 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:40.698 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:40.699 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:40.699 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:40.700 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:40.700 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:40.991 INFO:teuthology.orchestra.run.smithi022.stdout:Package 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:11:40.991 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do 2019-12-09T09:11:41.053 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install libcephfs-devel 2019-12-09T09:11:41.223 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:41.259 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:41.261 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:41.261 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:41.261 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:41.262 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:41.376 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:41.727 INFO:teuthology.orchestra.run.smithi035.stdout:Package 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:11:41.727 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do 2019-12-09T09:11:41.788 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install libcephfs-devel 2019-12-09T09:11:41.898 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:42.035 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:42.079 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:42.081 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:42.081 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:42.081 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:42.082 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:42.239 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:11:42.239 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:42.240 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:42.241 INFO:teuthology.orchestra.run.smithi022.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7900.g1351c81.el7 for package: 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:11:42.272 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:42.272 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:42.672 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:42.750 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:42.909 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:42.909 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:11:42.911 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:42.912 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:42.912 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository 2019-12-09T09:11:42.912 INFO:teuthology.orchestra.run.smithi022.stdout: Size 2019-12-09T09:11:42.912 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:42.912 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:11:42.913 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:11:42.913 INFO:teuthology.orchestra.run.smithi022.stdout:Installing for dependencies: 2019-12-09T09:11:42.913 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 153 k 2019-12-09T09:11:42.913 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:42.914 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:11:42.914 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:42.914 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package (+1 Dependent package) 2019-12-09T09:11:42.914 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:42.915 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 170 k 2019-12-09T09:11:42.915 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 679 k 2019-12-09T09:11:42.915 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:11:43.099 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:11:43.100 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:43.100 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:43.100 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7900.g1351c81.el7 for package: 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:11:43.134 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:43.134 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:43.320 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:11:43.320 INFO:teuthology.orchestra.run.smithi022.stdout:Total 412 kB/s | 170 kB 00:00 2019-12-09T09:11:43.327 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:11:43.341 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:11:43.342 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:11:43.342 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:11:43.526 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:43.759 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:43.759 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:11:43.760 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:43.761 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:43.761 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository 2019-12-09T09:11:43.761 INFO:teuthology.orchestra.run.smithi035.stdout: Size 2019-12-09T09:11:43.761 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:43.762 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:11:43.762 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:11:43.762 INFO:teuthology.orchestra.run.smithi035.stdout:Installing for dependencies: 2019-12-09T09:11:43.762 INFO:teuthology.orchestra.run.smithi035.stdout: librados-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 153 k 2019-12-09T09:11:43.763 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:43.763 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:11:43.763 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:43.763 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package (+1 Dependent package) 2019-12-09T09:11:43.763 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:43.764 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 170 k 2019-12-09T09:11:43.764 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 679 k 2019-12-09T09:11:43.764 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:11:44.075 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:11:44.139 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:11:44.139 INFO:teuthology.orchestra.run.smithi035.stdout:Total 448 kB/s | 170 kB 00:00 2019-12-09T09:11:44.142 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:11:44.154 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:11:44.158 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:11:44.158 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:11:44.319 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:11:44.424 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:11:44.777 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:11:44.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:11:44.884 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:44.884 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:11:44.884 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:44.884 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:44.885 INFO:teuthology.orchestra.run.smithi022.stdout:Dependency Installed: 2019-12-09T09:11:44.885 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:44.885 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:44.885 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:11:45.005 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:11:45.015 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install librados2 2019-12-09T09:11:45.119 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:11:45.224 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:45.263 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:45.264 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:45.265 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:45.265 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:45.266 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:45.934 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:Package 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:11:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do 2019-12-09T09:11:46.338 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install librbd1 2019-12-09T09:11:46.406 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:11:46.407 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:46.407 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:11:46.407 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:46.407 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:46.407 INFO:teuthology.orchestra.run.smithi035.stdout:Dependency Installed: 2019-12-09T09:11:46.407 INFO:teuthology.orchestra.run.smithi035.stdout: librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:46.407 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:46.408 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:11:46.530 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install librados2 2019-12-09T09:11:46.537 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:46.575 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:46.577 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:46.577 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:46.584 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:46.585 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:46.802 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:46.849 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:46.853 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:46.853 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:46.853 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:46.854 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:47.209 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:47.519 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:47.550 INFO:teuthology.orchestra.run.smithi022.stdout:Package 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:11:47.550 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do 2019-12-09T09:11:47.613 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install python-ceph 2019-12-09T09:11:47.785 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:47.837 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:47.838 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:47.838 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:47.838 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:47.838 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:47.875 INFO:teuthology.orchestra.run.smithi035.stdout:Package 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:11:47.876 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do 2019-12-09T09:11:47.936 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install librbd1 2019-12-09T09:11:48.153 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:48.197 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:48.198 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:48.199 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:48.199 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:48.199 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:48.459 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:48.803 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:11:48.803 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:48.804 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:48.864 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:49.194 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:49.214 INFO:teuthology.orchestra.run.smithi035.stdout:Package 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:11:49.214 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do 2019-12-09T09:11:49.276 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install python-ceph 2019-12-09T09:11:49.386 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:49.387 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:11:49.388 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:49.388 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:49.388 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository 2019-12-09T09:11:49.388 INFO:teuthology.orchestra.run.smithi022.stdout: Size 2019-12-09T09:11:49.389 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:49.389 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:11:49.389 INFO:teuthology.orchestra.run.smithi022.stdout: python-ceph-compat x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.2 k 2019-12-09T09:11:49.389 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:49.390 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:11:49.390 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:49.390 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2019-12-09T09:11:49.390 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:49.391 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 2.2 k 2019-12-09T09:11:49.391 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 0 2019-12-09T09:11:49.391 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:11:49.494 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:49.532 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:49.532 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:49.533 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:49.533 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:49.534 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:49.663 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:11:49.664 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:11:49.665 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:11:49.666 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:11:50.166 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:50.196 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:11:50.508 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:11:50.509 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:50.509 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:11:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:50.646 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:11:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:50.647 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:11:50.764 INFO:teuthology.orchestra.run.smithi022:> sudo yum -y install rbd-fuse 2019-12-09T09:11:50.901 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:51.045 INFO:teuthology.orchestra.run.smithi022.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:51.094 INFO:teuthology.orchestra.run.smithi022.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:51.094 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:51.095 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:11:51.095 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:51.095 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:51.095 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository 2019-12-09T09:11:51.096 INFO:teuthology.orchestra.run.smithi035.stdout: Size 2019-12-09T09:11:51.096 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:51.096 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:11:51.099 INFO:teuthology.orchestra.run.smithi035.stdout: python-ceph-compat x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.2 k 2019-12-09T09:11:51.099 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:51.100 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:11:51.100 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:51.101 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2019-12-09T09:11:51.101 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:51.101 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 2.2 k 2019-12-09T09:11:51.101 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 0 2019-12-09T09:11:51.101 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:11:51.102 INFO:teuthology.orchestra.run.smithi022.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:51.102 INFO:teuthology.orchestra.run.smithi022.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:51.102 INFO:teuthology.orchestra.run.smithi022.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:51.103 INFO:teuthology.orchestra.run.smithi022.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:51.366 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:11:51.367 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:11:51.369 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:11:51.370 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:11:51.771 INFO:teuthology.orchestra.run.smithi022.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:51.977 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:11:52.128 INFO:teuthology.orchestra.run.smithi022.stdout:Resolving Dependencies 2019-12-09T09:11:52.128 INFO:teuthology.orchestra.run.smithi022.stdout:--> Running transaction check 2019-12-09T09:11:52.128 INFO:teuthology.orchestra.run.smithi022.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:52.435 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:11:52.435 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:52.435 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:11:52.435 INFO:teuthology.orchestra.run.smithi035.stdout: python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:52.435 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:52.436 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:11:52.526 INFO:teuthology.orchestra.run.smithi022.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:52.549 INFO:teuthology.orchestra.run.smithi035:> sudo yum -y install rbd-fuse 2019-12-09T09:11:52.727 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:52.728 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies Resolved 2019-12-09T09:11:52.729 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:52.729 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:52.729 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2019-12-09T09:11:52.729 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:52.729 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2019-12-09T09:11:52.729 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 76 k 2019-12-09T09:11:52.729 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:52.730 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2019-12-09T09:11:52.730 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2019-12-09T09:11:52.730 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2019-12-09T09:11:52.730 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:52.730 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 76 k 2019-12-09T09:11:52.730 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 200 k 2019-12-09T09:11:52.730 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading packages: 2019-12-09T09:11:52.810 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:11:52.852 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:11:52.853 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:11:52.854 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:11:52.854 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:11:52.854 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:11:53.022 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2019-12-09T09:11:53.038 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2019-12-09T09:11:53.040 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded 2019-12-09T09:11:53.040 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2019-12-09T09:11:53.516 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:11:53.869 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:11:53.873 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:11:53.873 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:11:53.995 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:11:54.264 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:11:54.457 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:54.457 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:11:54.458 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:54.458 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:54.458 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2019-12-09T09:11:54.458 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:54.458 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:11:54.458 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 76 k 2019-12-09T09:11:54.458 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:54.459 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:11:54.459 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:11:54.459 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2019-12-09T09:11:54.459 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:54.459 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 76 k 2019-12-09T09:11:54.459 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 200 k 2019-12-09T09:11:54.459 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:11:54.500 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:11:54.501 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:54.501 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2019-12-09T09:11:54.501 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:54.501 INFO:teuthology.orchestra.run.smithi022.stdout: 2019-12-09T09:11:54.501 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2019-12-09T09:11:54.621 DEBUG:teuthology.parallel:result is None 2019-12-09T09:11:54.739 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:11:54.749 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:11:54.750 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:11:54.750 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:11:55.539 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:11:56.167 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:11:56.167 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:56.167 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:11:56.168 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:56.168 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:11:56.168 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:11:56.293 DEBUG:teuthology.parallel:result is None 2019-12-09T09:11:56.293 INFO:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2019-12-09T09:11:56.447 INFO:teuthology.orchestra.run.smithi035.stdout:CentOS 2019-12-09T09:11:56.448 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:11:56.448 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:11:56.611 INFO:teuthology.orchestra.run.smithi035:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-09T09:11:56.755 INFO:teuthology.orchestra.run.smithi035.stdout:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:56.756 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7900.g1351c81.el7 2019-12-09T09:11:56.756 INFO:teuthology.task.install:The correct ceph version 15.0.0-7900.g1351c81 is installed. 2019-12-09T09:11:56.756 INFO:teuthology.orchestra.run.smithi022:> sudo lsb_release -is 2019-12-09T09:11:56.907 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS 2019-12-09T09:11:56.910 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:11:56.911 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:11:57.093 INFO:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-09T09:11:57.234 INFO:teuthology.orchestra.run.smithi022.stdout:15.0.0-7900.g1351c81.el7 2019-12-09T09:11:57.234 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7900.g1351c81.el7 2019-12-09T09:11:57.235 INFO:teuthology.task.install:The correct ceph version 15.0.0-7900.g1351c81 is installed. 2019-12-09T09:11:57.235 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2019-12-09T09:11:57.235 INFO:teuthology.orchestra.run.smithi035:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-09T09:11:57.360 INFO:teuthology.orchestra.run.smithi022:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-09T09:11:57.488 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2019-12-09T09:11:57.489 INFO:teuthology.orchestra.run.smithi035:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-09T09:11:57.618 INFO:teuthology.orchestra.run.smithi035:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-09T09:11:57.732 INFO:teuthology.orchestra.run.smithi022:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-09T09:11:57.862 INFO:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-09T09:11:57.971 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2019-12-09T09:11:57.972 INFO:teuthology.orchestra.run.smithi035:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-09T09:11:58.098 INFO:teuthology.orchestra.run.smithi035:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-09T09:11:58.204 INFO:teuthology.orchestra.run.smithi022:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-09T09:11:58.330 INFO:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-09T09:11:58.427 INFO:teuthology.run_tasks:Running task ceph... 2019-12-09T09:11:58.508 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2019-12-09T09:11:58.509 INFO:teuthology.orchestra.run.smithi022:> sudo chmod 777 /var/log/ceph 2019-12-09T09:11:58.513 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 777 /var/log/ceph 2019-12-09T09:11:58.635 INFO:tasks.ceph:Disabling ceph logrotate... 2019-12-09T09:11:58.635 INFO:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/logrotate.d/ceph 2019-12-09T09:11:58.639 INFO:teuthology.orchestra.run.smithi035:> sudo rm -f -- /etc/logrotate.d/ceph 2019-12-09T09:11:58.749 INFO:tasks.ceph:Creating extra log directories... 2019-12-09T09:11:58.749 INFO:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-12-09T09:11:58.753 INFO:teuthology.orchestra.run.smithi035:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-12-09T09:11:58.870 INFO:tasks.ceph:Creating ceph cluster ceph... 2019-12-09T09:11:58.870 INFO:tasks.ceph:config {'mon_bind_addrvec': True, 'block_journal': None, 'fs': 'xfs', 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'skip_mgr_daemons': False, 'cpu_profile': set([]), 'mkfs_options': None, 'tmpfs_journal': None, 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mount_options': None, 'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'debug ms': 1, 'debug journal': 20, 'osd fast shutdown': False, 'debug osd': 25, 'osd objectstore': 'filestore', 'osd sloppy crc': True, 'debug filestore': 20, 'osd shutdown pgref assert': True}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}} 2019-12-09T09:11:58.870 INFO:tasks.ceph:ctx.config {'kernel': {'sha1': 'distro', 'kdb': True}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'sha1': 'distro', 'kdb': True}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh']}}}, {'ragweed': {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}}}, {'s3tests-java': {'client.0': {'force-repo': 'https://github.com/ceph/java_s3tests.git', 'force-branch': 'master'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master'}}}, {'swift': {'client.0': {'rgw_server': 'client.0'}}}], 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.22738', 'owner': 'scheduled_ofriedma@teuthology', 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'targets': {'smithi035.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCotl1Z+PSkiOfwNCfW9Np2KP4HeR18WJ23R7YL66Fu6Moyya5PRbhv4wvQtBXiof0aAQnF55aHkOBO4MaeqynaSwnnjhR1eoT6LRT1Jakg1egN6IMLZnBaBJyfPXdGy/DPfnhghNunyUGAhDwREosZXd0Ur6BDcpg2NRNzhR0Rub4oZInZ7yMqbTJsEQTtguXEfkR4aG02dfcHQecQ5BK/10zn7uBNrRZZWLGYTwCosT5M9hbs4N8y9oEmBDYTtG6ZPCL5OgVdrNDuQykuA2IIqT0aq35hOhm8WJSaGnoMwirsjVSbmGNoB4ZCEzY45RGB+mEQK/lES39B60MChtnv', 'smithi022.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4hazoKCZRee8MFRtc7S7yQdf1IxkTxCzrh/vNl69HpJJxXzCBQ3yBjc+QdjJYo2us10/vk+/JdIdHhuwEqQ/6lEJZMLsj6veaoX5Ugr8F9AHPD2K9kv3g9T3rURL/oM2oGGFwNe+lc8BdsVyEqVt3VxmkLniK5kci5rwkFCf8xek8qt3AmMdEMZBMsMPclJ9NW2Xw4KCvYMXoTbPpVs5S2bFHTCua9RBjRPqRr+3vBnLx3PKyIIMaT7oNerbszXORGFQWH3RNbLfinhFQIonxR+7BqezG+kn2d8pv38SJPT0Pf84WSN1NN1drfybCIVKXkDpP5oT9ZCWtVU0E6ZDX'}, 'job_id': '4584190', 'tube': 'smithi', 'suite_branch': 'wip-ofriedma-dynamic-disable', 'priority': 60, 'first_in_suite': False, 'branch': 'wip-ofriedma-dynamic-disable', 'suite': 'rgw', 'email': 'ofriedma@redhat.com', 'archive_path': '/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584190', 'description': 'rgw/verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml}', 'nuke-on-error': True, 'repo': 'https://github.com/ceph/ceph-ci.git', 'last_in_suite': False, 'machine_type': 'smithi', 'teuthology_branch': 'master', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'name': 'ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'os_type': 'centos', 'overrides': {'ceph': {'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'debug ms': 1, 'debug journal': 20, 'osd fast shutdown': False, 'debug osd': 25, 'osd objectstore': 'filestore', 'osd sloppy crc': True, 'debug filestore': 20, 'osd shutdown pgref assert': True}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'fs': 'xfs', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}, 's3tests': {'slow_backend': True}, 'ceph-deploy': {'fs': 'xfs', 'filestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'osd sloppy crc': True, 'osd objectstore': 'filestore'}}}, 'install': {'ceph': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}}, 'rgw': {'frontend': 'civetweb', 'compression type': 'random', 'storage classes': 'LUKEWARM, FROZEN', 'ec-data-pool': True}, 'admin_socket': {'branch': 'wip-ofriedma-dynamic-disable'}, 'workunit': {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'branch': 'wip-ofriedma-dynamic-disable'}}} 2019-12-09T09:11:58.871 INFO:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-12-09T09:11:58.880 INFO:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-12-09T09:11:58.971 INFO:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/run/ceph 2019-12-09T09:11:58.975 INFO:teuthology.orchestra.run.smithi035:> sudo install -d -m0777 -- /var/run/ceph 2019-12-09T09:11:59.170 DEBUG:teuthology.orchestra.remote:smithi035:/scratch_devs is 72B 2019-12-09T09:11:59.247 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:11:59.247 INFO:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-09T09:11:59.313 INFO:teuthology.orchestra.run.smithi035.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-09T09:11:59.314 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:11:59.314 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 5h/5d Inode: 100956 Links: 1 2019-12-09T09:11:59.314 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:11:59.314 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:11:59.314 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2019-12-09 09:10:45.610457824 +0000 2019-12-09T09:11:59.314 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2019-12-09 09:08:56.578194790 +0000 2019-12-09T09:11:59.315 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2019-12-09 09:08:56.578194790 +0000 2019-12-09T09:11:59.315 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2019-12-09T09:11:59.337 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2019-12-09T09:11:59.338 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2019-12-09T09:11:59.338 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes (512 B) copied, 0.000354479 s, 1.4 MB/s 2019-12-09T09:11:59.343 INFO:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-09T09:11:59.422 INFO:teuthology.orchestra.run.smithi035.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-09T09:11:59.422 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:11:59.422 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 5h/5d Inode: 107621 Links: 1 2019-12-09T09:11:59.422 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:11:59.422 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:11:59.423 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2019-12-09 09:10:45.610457824 +0000 2019-12-09T09:11:59.423 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2019-12-09 09:08:57.069183848 +0000 2019-12-09T09:11:59.423 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2019-12-09 09:08:57.069183848 +0000 2019-12-09T09:11:59.423 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2019-12-09T09:11:59.443 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2019-12-09T09:11:59.443 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2019-12-09T09:11:59.444 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes (512 B) copied, 0.000386771 s, 1.3 MB/s 2019-12-09T09:11:59.448 INFO:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-09T09:11:59.516 INFO:teuthology.orchestra.run.smithi035.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-09T09:11:59.517 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:11:59.517 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 5h/5d Inode: 100994 Links: 1 2019-12-09T09:11:59.517 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:11:59.517 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:11:59.517 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2019-12-09 09:10:45.610457824 +0000 2019-12-09T09:11:59.517 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2019-12-09 09:08:57.589172255 +0000 2019-12-09T09:11:59.518 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2019-12-09 09:08:57.589172255 +0000 2019-12-09T09:11:59.518 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2019-12-09T09:11:59.540 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2019-12-09T09:11:59.540 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2019-12-09T09:11:59.541 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes (512 B) copied, 0.000350626 s, 1.5 MB/s 2019-12-09T09:11:59.544 INFO:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-09T09:11:59.612 INFO:teuthology.orchestra.run.smithi035.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-09T09:11:59.613 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:11:59.613 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 5h/5d Inode: 107657 Links: 1 2019-12-09T09:11:59.613 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:11:59.614 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:11:59.614 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2019-12-09 09:10:45.611457724 +0000 2019-12-09T09:11:59.614 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2019-12-09 09:08:58.064161673 +0000 2019-12-09T09:11:59.614 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2019-12-09 09:08:58.064161673 +0000 2019-12-09T09:11:59.614 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2019-12-09T09:11:59.634 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2019-12-09T09:11:59.635 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2019-12-09T09:11:59.635 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes (512 B) copied, 0.000508452 s, 1.0 MB/s 2019-12-09T09:11:59.639 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-12-09T09:11:59.641 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:11:59.641 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-12-09T09:11:59.641 INFO:tasks.ceph:dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:11:59.735 DEBUG:teuthology.orchestra.remote:smithi022:/scratch_devs is 72B 2019-12-09T09:11:59.827 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:11:59.828 INFO:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-09T09:11:59.915 INFO:teuthology.orchestra.run.smithi022.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-09T09:11:59.915 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:11:59.915 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 107697 Links: 1 2019-12-09T09:11:59.916 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:11:59.916 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:11:59.916 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2019-12-09 09:10:48.810109160 +0000 2019-12-09T09:11:59.916 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2019-12-09 09:08:55.623702169 +0000 2019-12-09T09:11:59.916 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2019-12-09 09:08:55.623702169 +0000 2019-12-09T09:11:59.916 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2019-12-09T09:11:59.944 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2019-12-09T09:11:59.944 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2019-12-09T09:11:59.944 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes (512 B) copied, 0.000398219 s, 1.3 MB/s 2019-12-09T09:11:59.951 INFO:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-09T09:12:00.022 INFO:teuthology.orchestra.run.smithi022.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-09T09:12:00.023 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:12:00.023 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 108696 Links: 1 2019-12-09T09:12:00.023 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:12:00.023 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:12:00.023 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2019-12-09 09:10:48.810109160 +0000 2019-12-09T09:12:00.024 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2019-12-09 09:08:56.131697586 +0000 2019-12-09T09:12:00.024 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2019-12-09 09:08:56.131697586 +0000 2019-12-09T09:12:00.024 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2019-12-09T09:12:00.049 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2019-12-09T09:12:00.049 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2019-12-09T09:12:00.050 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes (512 B) copied, 0.000422937 s, 1.2 MB/s 2019-12-09T09:12:00.053 INFO:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-09T09:12:00.116 INFO:teuthology.orchestra.run.smithi022.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-09T09:12:00.116 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:12:00.117 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 106653 Links: 1 2019-12-09T09:12:00.117 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:12:00.117 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:12:00.117 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2019-12-09 09:10:48.810109160 +0000 2019-12-09T09:12:00.117 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2019-12-09 09:08:56.619693182 +0000 2019-12-09T09:12:00.118 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2019-12-09 09:08:56.619693182 +0000 2019-12-09T09:12:00.118 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2019-12-09T09:12:00.139 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2019-12-09T09:12:00.139 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2019-12-09T09:12:00.152 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes (512 B) copied, 0.000411793 s, 1.2 MB/s 2019-12-09T09:12:00.153 INFO:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-09T09:12:00.243 INFO:teuthology.orchestra.run.smithi022.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-09T09:12:00.243 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:12:00.243 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 104068 Links: 1 2019-12-09T09:12:00.243 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:12:00.244 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:12:00.244 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2019-12-09 09:10:48.810109160 +0000 2019-12-09T09:12:00.244 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2019-12-09 09:08:57.109688761 +0000 2019-12-09T09:12:00.244 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2019-12-09 09:08:57.109688761 +0000 2019-12-09T09:12:00.244 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2019-12-09T09:12:00.267 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2019-12-09T09:12:00.267 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2019-12-09T09:12:00.267 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes (512 B) copied, 0.000360649 s, 1.4 MB/s 2019-12-09T09:12:00.272 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-12-09T09:12:00.272 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:12:00.272 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-12-09T09:12:00.272 INFO:tasks.ceph:dev map: {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:12:00.272 INFO:tasks.ceph:Generating config... 2019-12-09T09:12:00.274 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2019-12-09T09:12:00.274 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2019-12-09T09:12:00.274 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2019-12-09T09:12:00.274 INFO:tasks.ceph:[global] mon client directed command retry = 5 2019-12-09T09:12:00.274 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2019-12-09T09:12:00.275 INFO:tasks.ceph:[osd] debug ms = 1 2019-12-09T09:12:00.275 INFO:tasks.ceph:[osd] debug journal = 20 2019-12-09T09:12:00.275 INFO:tasks.ceph:[osd] osd fast shutdown = False 2019-12-09T09:12:00.275 INFO:tasks.ceph:[osd] debug osd = 25 2019-12-09T09:12:00.275 INFO:tasks.ceph:[osd] osd objectstore = filestore 2019-12-09T09:12:00.275 INFO:tasks.ceph:[osd] osd sloppy crc = True 2019-12-09T09:12:00.275 INFO:tasks.ceph:[osd] debug filestore = 20 2019-12-09T09:12:00.276 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2019-12-09T09:12:00.276 INFO:tasks.ceph:[mon] debug mon = 20 2019-12-09T09:12:00.276 INFO:tasks.ceph:[mon] debug paxos = 20 2019-12-09T09:12:00.276 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2019-12-09T09:12:00.276 INFO:tasks.ceph:[mon] debug ms = 1 2019-12-09T09:12:00.276 INFO:tasks.ceph:[client] debug rgw = 20 2019-12-09T09:12:00.276 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2019-12-09T09:12:00.276 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2019-12-09T09:12:00.277 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2019-12-09T09:12:00.277 INFO:tasks.ceph:[client] rgw lc debug interval = 10 2019-12-09T09:12:00.277 INFO:tasks.ceph:Setting up mon.a... 2019-12-09T09:12:00.277 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2019-12-09T09:12:00.465 INFO:teuthology.orchestra.run.smithi035.stdout:creating /etc/ceph/ceph.keyring 2019-12-09T09:12:00.470 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2019-12-09T09:12:00.631 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T09:12:00.739 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.35'), ('mon.c', '[v2:172.21.15.35:3301,v1:172.21.15.35:6790]'), ('mon.b', '172.21.15.22')] 2019-12-09T09:12:00.739 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.35 --addv c '[v2:172.21.15.35:3301,v1:172.21.15.35:6790]' --add b 172.21.15.22 --print /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:00.852 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:00.852 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: generated fsid 92300541-fe50-4dfa-a6fe-782b29b49e98 2019-12-09T09:12:00.852 INFO:teuthology.orchestra.run.smithi035.stdout:epoch 0 2019-12-09T09:12:00.853 INFO:teuthology.orchestra.run.smithi035.stdout:fsid 92300541-fe50-4dfa-a6fe-782b29b49e98 2019-12-09T09:12:00.853 INFO:teuthology.orchestra.run.smithi035.stdout:last_changed 2019-12-09T09:12:00.838779+0000 2019-12-09T09:12:00.853 INFO:teuthology.orchestra.run.smithi035.stdout:created 2019-12-09T09:12:00.838779+0000 2019-12-09T09:12:00.853 INFO:teuthology.orchestra.run.smithi035.stdout:min_mon_release 0 (unknown) 2019-12-09T09:12:00.853 INFO:teuthology.orchestra.run.smithi035.stdout:0: [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] mon.a 2019-12-09T09:12:00.853 INFO:teuthology.orchestra.run.smithi035.stdout:1: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.b 2019-12-09T09:12:00.853 INFO:teuthology.orchestra.run.smithi035.stdout:2: [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] mon.c 2019-12-09T09:12:00.854 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2019-12-09T09:12:00.856 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 92300541-fe50-4dfa-a6fe-782b29b49e98... 2019-12-09T09:12:00.857 INFO:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-12-09T09:12:00.862 INFO:teuthology.orchestra.run.smithi035:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-12-09T09:12:01.083 INFO:tasks.ceph:Creating admin key on mon.a... 2019-12-09T09:12:01.083 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2019-12-09T09:12:01.271 INFO:tasks.ceph:Copying monmap to all nodes... 2019-12-09T09:12:01.350 DEBUG:teuthology.orchestra.remote:smithi035:/etc/ceph/ceph.keyring is 206B 2019-12-09T09:12:01.526 DEBUG:teuthology.orchestra.remote:smithi035:/home/ubuntu/cephtest/ceph.monmap is 379B 2019-12-09T09:12:01.629 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:12:01.630 INFO:teuthology.orchestra.run.smithi035:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T09:12:01.783 INFO:teuthology.orchestra.run.smithi035:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:01.864 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi022.front.sepia.ceph.com 2019-12-09T09:12:01.864 INFO:teuthology.orchestra.run.smithi022:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T09:12:02.029 INFO:teuthology.orchestra.run.smithi022:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:02.111 INFO:tasks.ceph:Setting up mon nodes... 2019-12-09T09:12:02.112 INFO:tasks.ceph:Setting up mgr nodes... 2019-12-09T09:12:02.112 INFO:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2019-12-09T09:12:02.298 INFO:teuthology.orchestra.run.smithi035.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2019-12-09T09:12:02.303 INFO:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2019-12-09T09:12:02.509 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2019-12-09T09:12:02.513 INFO:tasks.ceph:Setting up mds nodes... 2019-12-09T09:12:02.513 INFO:tasks.ceph_client:Setting up client nodes... 2019-12-09T09:12:02.514 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2019-12-09T09:12:02.663 INFO:teuthology.orchestra.run.smithi035.stdout:creating /etc/ceph/ceph.client.0.keyring 2019-12-09T09:12:02.709 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2019-12-09T09:12:02.862 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.1.keyring 2019-12-09T09:12:02.895 INFO:tasks.ceph:Running mkfs on osd nodes... 2019-12-09T09:12:02.895 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi035.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'}} 2019-12-09T09:12:02.896 INFO:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2019-12-09T09:12:03.025 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:12:03.025 INFO:tasks.ceph:{} 2019-12-09T09:12:03.025 INFO:tasks.ceph:osd.0 2019-12-09T09:12:03.025 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:12:03.026 INFO:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-12-09T09:12:03.640 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:12:03.640 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:12:03.640 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:12:03.640 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:12:03.641 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:12:03.641 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:12:03.641 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:12:03.641 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:12:03.641 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:12:03.645 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2019-12-09T09:12:03.645 INFO:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0 2019-12-09T09:12:03.793 INFO:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2019-12-09T09:12:03.907 INFO:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2019-12-09T09:12:04.016 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:12:04.016 INFO:tasks.ceph:{} 2019-12-09T09:12:04.016 INFO:tasks.ceph:osd.1 2019-12-09T09:12:04.016 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:12:04.016 INFO:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-12-09T09:12:04.656 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:12:04.656 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:12:04.656 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:12:04.656 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:12:04.657 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:12:04.657 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:12:04.657 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:12:04.657 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:12:04.659 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:12:04.659 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2019-12-09T09:12:04.659 INFO:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2019-12-09T09:12:04.811 INFO:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2019-12-09T09:12:04.926 INFO:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2019-12-09T09:12:05.026 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:12:05.026 INFO:tasks.ceph:{} 2019-12-09T09:12:05.027 INFO:tasks.ceph:osd.2 2019-12-09T09:12:05.027 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:12:05.027 INFO:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-12-09T09:12:05.646 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:12:05.646 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:12:05.646 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:12:05.646 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:12:05.646 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:12:05.646 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:12:05.646 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:12:05.647 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:12:05.647 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:12:05.654 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2019-12-09T09:12:05.655 INFO:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2019-12-09T09:12:05.783 INFO:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2019-12-09T09:12:05.873 INFO:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2019-12-09T09:12:05.950 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:12:05.950 INFO:tasks.ceph:{} 2019-12-09T09:12:05.950 INFO:tasks.ceph:osd.3 2019-12-09T09:12:05.951 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:12:05.951 INFO:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-12-09T09:12:06.526 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:12:06.526 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:12:06.526 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:12:06.527 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:12:06.527 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:12:06.527 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:12:06.527 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:12:06.528 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:12:06.528 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:12:06.529 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2019-12-09T09:12:06.529 INFO:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2019-12-09T09:12:06.651 INFO:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2019-12-09T09:12:06.760 INFO:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:06.902 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:06.887+0000 7f51efef9b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2019-12-09T09:12:06.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:06.888+0000 7f51efef9b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2019-12-09T09:12:06.917 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:06.902+0000 7f51efef9b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:06.917 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:06.902+0000 7f51efef9b40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:12:07.055 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.039+0000 7f51efef9b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:07.055 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.040+0000 7f51efef9b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:07.055 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.040+0000 7f51efef9b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:07.068 INFO:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2019-12-09T09:12:07.164 INFO:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:07.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.269+0000 7f5d89892b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2019-12-09T09:12:07.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.270+0000 7f5d89892b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2019-12-09T09:12:07.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.282+0000 7f5d89892b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:07.298 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.282+0000 7f5d89892b40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:12:07.434 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.418+0000 7f5d89892b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:07.434 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.419+0000 7f5d89892b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:07.434 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.419+0000 7f5d89892b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:07.451 INFO:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2019-12-09T09:12:07.546 INFO:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:07.665 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.650+0000 7f3c41f72b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2019-12-09T09:12:07.665 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.650+0000 7f3c41f72b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2019-12-09T09:12:07.679 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.664+0000 7f3c41f72b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:07.679 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.664+0000 7f3c41f72b40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:12:07.811 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.796+0000 7f3c41f72b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:07.812 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.796+0000 7f3c41f72b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:07.812 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:07.796+0000 7f3c41f72b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:07.824 INFO:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2019-12-09T09:12:07.918 INFO:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:08.049 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:08.033+0000 7fd8dd0d9b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2019-12-09T09:12:08.049 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:08.033+0000 7fd8dd0d9b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2019-12-09T09:12:08.063 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:08.047+0000 7fd8dd0d9b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:08.063 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:08.047+0000 7fd8dd0d9b40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:12:08.197 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:08.181+0000 7fd8dd0d9b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:08.197 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:08.182+0000 7fd8dd0d9b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:08.197 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:12:08.182+0000 7fd8dd0d9b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:08.210 INFO:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2019-12-09T09:12:08.314 INFO:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2019-12-09T09:12:08.431 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:12:08.431 INFO:tasks.ceph:{} 2019-12-09T09:12:08.431 INFO:tasks.ceph:osd.4 2019-12-09T09:12:08.432 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com 2019-12-09T09:12:08.432 INFO:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-12-09T09:12:09.021 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:12:09.021 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:12:09.021 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:12:09.021 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:12:09.021 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:12:09.021 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:12:09.022 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:12:09.022 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:12:09.022 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:12:09.023 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2019-12-09T09:12:09.024 INFO:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4 2019-12-09T09:12:09.218 INFO:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2019-12-09T09:12:09.333 INFO:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2019-12-09T09:12:09.432 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:12:09.432 INFO:tasks.ceph:{} 2019-12-09T09:12:09.433 INFO:tasks.ceph:osd.5 2019-12-09T09:12:09.433 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com 2019-12-09T09:12:09.433 INFO:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-12-09T09:12:09.993 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:12:09.993 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:12:09.994 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:12:09.994 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:12:09.994 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:12:09.994 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:12:09.994 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:12:09.994 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:12:09.995 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:12:09.996 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2019-12-09T09:12:09.996 INFO:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-5 2019-12-09T09:12:10.142 INFO:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2019-12-09T09:12:10.260 INFO:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2019-12-09T09:12:10.358 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:12:10.358 INFO:tasks.ceph:{} 2019-12-09T09:12:10.358 INFO:tasks.ceph:osd.6 2019-12-09T09:12:10.358 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com 2019-12-09T09:12:10.358 INFO:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-12-09T09:12:10.949 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:12:10.949 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:12:10.949 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:12:10.949 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:12:10.949 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:12:10.949 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:12:10.950 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:12:10.950 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:12:10.950 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:12:10.952 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2019-12-09T09:12:10.952 INFO:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2019-12-09T09:12:11.109 INFO:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2019-12-09T09:12:11.223 INFO:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2019-12-09T09:12:11.323 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:12:11.323 INFO:tasks.ceph:{} 2019-12-09T09:12:11.323 INFO:tasks.ceph:osd.7 2019-12-09T09:12:11.324 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com 2019-12-09T09:12:11.324 INFO:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-12-09T09:12:11.891 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:12:11.891 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:12:11.891 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:12:11.892 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:12:11.892 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:12:11.892 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:12:11.892 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:12:11.892 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:12:11.892 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:12:11.894 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2019-12-09T09:12:11.894 INFO:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-7 2019-12-09T09:12:12.041 INFO:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2019-12-09T09:12:12.150 INFO:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:12.306 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:12.306+0000 7fec92754b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2019-12-09T09:12:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:12.312+0000 7fec92754b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2019-12-09T09:12:12.350 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:12.350+0000 7fec92754b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:12.350 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:12.350+0000 7fec92754b40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:12:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:12.608+0000 7fec92754b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:12.610+0000 7fec92754b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:12.610+0000 7fec92754b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:12.641 INFO:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2019-12-09T09:12:12.760 INFO:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:12.912 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:12.912+0000 7f448310bb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2019-12-09T09:12:12.917 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:12.917+0000 7f448310bb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2019-12-09T09:12:12.958 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:12.958+0000 7f448310bb40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:12.958 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:12.958+0000 7f448310bb40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:12:13.173 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.173+0000 7f448310bb40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:13.173 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.173+0000 7f448310bb40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:13.173 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.173+0000 7f448310bb40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:13.186 INFO:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2019-12-09T09:12:13.291 INFO:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:13.440 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.440+0000 7f47e5a80b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2019-12-09T09:12:13.441 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.441+0000 7f47e5a80b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2019-12-09T09:12:13.456 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.457+0000 7f47e5a80b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:13.457 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.457+0000 7f47e5a80b40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:12:13.595 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.595+0000 7f47e5a80b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:13.595 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.595+0000 7f47e5a80b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:13.595 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.595+0000 7f47e5a80b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:13.608 INFO:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2019-12-09T09:12:13.722 INFO:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:13.864 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.864+0000 7f9861ef9b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2019-12-09T09:12:13.865 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.865+0000 7f9861ef9b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2019-12-09T09:12:13.878 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.879+0000 7f9861ef9b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:13.878 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:13.879+0000 7f9861ef9b40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:12:14.019 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:14.019+0000 7f9861ef9b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:14.020 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:14.019+0000 7f9861ef9b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:14.020 INFO:teuthology.orchestra.run.smithi022.stderr:2019-12-09T09:12:14.019+0000 7f9861ef9b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:12:14.031 INFO:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2019-12-09T09:12:14.136 INFO:tasks.ceph:Reading keys from all nodes... 2019-12-09T09:12:14.137 INFO:teuthology.orchestra.run.smithi035:> mktemp 2019-12-09T09:12:14.229 INFO:teuthology.orchestra.run.smithi035.stdout:/tmp/tmp.r1OWLCtgB6 2019-12-09T09:12:14.230 INFO:teuthology.orchestra.run.smithi035:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.r1OWLCtgB6 2019-12-09T09:12:14.341 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 0666 /tmp/tmp.r1OWLCtgB6 2019-12-09T09:12:14.559 DEBUG:teuthology.orchestra.remote:smithi035:/tmp/tmp.r1OWLCtgB6 is 56B 2019-12-09T09:12:14.644 INFO:teuthology.orchestra.run.smithi035:> rm -fr /tmp/tmp.r1OWLCtgB6 2019-12-09T09:12:14.726 INFO:teuthology.orchestra.run.smithi035:> mktemp 2019-12-09T09:12:14.825 INFO:teuthology.orchestra.run.smithi035.stdout:/tmp/tmp.PLrTKzwldT 2019-12-09T09:12:14.825 INFO:teuthology.orchestra.run.smithi035:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.PLrTKzwldT 2019-12-09T09:12:14.935 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 0666 /tmp/tmp.PLrTKzwldT 2019-12-09T09:12:15.112 DEBUG:teuthology.orchestra.remote:smithi035:/tmp/tmp.PLrTKzwldT is 56B 2019-12-09T09:12:15.188 INFO:teuthology.orchestra.run.smithi035:> rm -fr /tmp/tmp.PLrTKzwldT 2019-12-09T09:12:15.254 INFO:teuthology.orchestra.run.smithi035:> mktemp 2019-12-09T09:12:15.385 INFO:teuthology.orchestra.run.smithi035.stdout:/tmp/tmp.ShtslrHjcm 2019-12-09T09:12:15.386 INFO:teuthology.orchestra.run.smithi035:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.ShtslrHjcm 2019-12-09T09:12:15.508 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 0666 /tmp/tmp.ShtslrHjcm 2019-12-09T09:12:15.680 DEBUG:teuthology.orchestra.remote:smithi035:/tmp/tmp.ShtslrHjcm is 56B 2019-12-09T09:12:15.749 INFO:teuthology.orchestra.run.smithi035:> rm -fr /tmp/tmp.ShtslrHjcm 2019-12-09T09:12:15.807 INFO:teuthology.orchestra.run.smithi035:> mktemp 2019-12-09T09:12:15.890 INFO:teuthology.orchestra.run.smithi035.stdout:/tmp/tmp.xkiMonp0oa 2019-12-09T09:12:15.890 INFO:teuthology.orchestra.run.smithi035:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.xkiMonp0oa 2019-12-09T09:12:16.005 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 0666 /tmp/tmp.xkiMonp0oa 2019-12-09T09:12:16.194 DEBUG:teuthology.orchestra.remote:smithi035:/tmp/tmp.xkiMonp0oa is 56B 2019-12-09T09:12:16.271 INFO:teuthology.orchestra.run.smithi035:> rm -fr /tmp/tmp.xkiMonp0oa 2019-12-09T09:12:16.356 INFO:teuthology.orchestra.run.smithi035:> mktemp 2019-12-09T09:12:16.442 INFO:teuthology.orchestra.run.smithi035.stdout:/tmp/tmp.anhPKRMZrd 2019-12-09T09:12:16.442 INFO:teuthology.orchestra.run.smithi035:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.anhPKRMZrd 2019-12-09T09:12:16.559 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 0666 /tmp/tmp.anhPKRMZrd 2019-12-09T09:12:16.769 DEBUG:teuthology.orchestra.remote:smithi035:/tmp/tmp.anhPKRMZrd is 56B 2019-12-09T09:12:16.863 INFO:teuthology.orchestra.run.smithi035:> rm -fr /tmp/tmp.anhPKRMZrd 2019-12-09T09:12:16.943 INFO:teuthology.orchestra.run.smithi022:> mktemp 2019-12-09T09:12:17.033 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.d3vadTaEkf 2019-12-09T09:12:17.034 INFO:teuthology.orchestra.run.smithi022:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.d3vadTaEkf 2019-12-09T09:12:17.151 INFO:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.d3vadTaEkf 2019-12-09T09:12:17.340 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.d3vadTaEkf is 56B 2019-12-09T09:12:17.411 INFO:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.d3vadTaEkf 2019-12-09T09:12:17.480 INFO:teuthology.orchestra.run.smithi022:> mktemp 2019-12-09T09:12:17.545 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.dbmdSWKDhT 2019-12-09T09:12:17.546 INFO:teuthology.orchestra.run.smithi022:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.dbmdSWKDhT 2019-12-09T09:12:17.635 INFO:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.dbmdSWKDhT 2019-12-09T09:12:17.786 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.dbmdSWKDhT is 56B 2019-12-09T09:12:17.865 INFO:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.dbmdSWKDhT 2019-12-09T09:12:17.941 INFO:teuthology.orchestra.run.smithi022:> mktemp 2019-12-09T09:12:18.071 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.fyVVMeRhkx 2019-12-09T09:12:18.071 INFO:teuthology.orchestra.run.smithi022:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.fyVVMeRhkx 2019-12-09T09:12:18.176 INFO:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.fyVVMeRhkx 2019-12-09T09:12:18.387 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.fyVVMeRhkx is 56B 2019-12-09T09:12:18.478 INFO:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.fyVVMeRhkx 2019-12-09T09:12:18.550 INFO:teuthology.orchestra.run.smithi022:> mktemp 2019-12-09T09:12:18.620 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.xIrZ3agsZp 2019-12-09T09:12:18.621 INFO:teuthology.orchestra.run.smithi022:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.xIrZ3agsZp 2019-12-09T09:12:18.729 INFO:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.xIrZ3agsZp 2019-12-09T09:12:18.926 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.xIrZ3agsZp is 56B 2019-12-09T09:12:19.003 INFO:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.xIrZ3agsZp 2019-12-09T09:12:19.086 INFO:teuthology.orchestra.run.smithi022:> mktemp 2019-12-09T09:12:19.160 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.6h7YSjNwsa 2019-12-09T09:12:19.160 INFO:teuthology.orchestra.run.smithi022:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.6h7YSjNwsa 2019-12-09T09:12:19.262 INFO:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.6h7YSjNwsa 2019-12-09T09:12:19.455 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.6h7YSjNwsa is 56B 2019-12-09T09:12:19.543 INFO:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.6h7YSjNwsa 2019-12-09T09:12:19.728 DEBUG:teuthology.orchestra.remote:smithi035:/etc/ceph/ceph.client.0.keyring is 59B 2019-12-09T09:12:19.917 DEBUG:teuthology.orchestra.remote:smithi022:/etc/ceph/ceph.client.1.keyring is 59B 2019-12-09T09:12:20.009 INFO:tasks.ceph:Adding keys to all mons... 2019-12-09T09:12:20.009 INFO:teuthology.orchestra.run.smithi022:> sudo tee -a /etc/ceph/ceph.keyring 2019-12-09T09:12:20.013 INFO:teuthology.orchestra.run.smithi035:> sudo tee -a /etc/ceph/ceph.keyring 2019-12-09T09:12:20.123 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.y] 2019-12-09T09:12:20.123 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDiD+5dFND0EBAAiD4RLNXaeSeodoG95QywcQ== 2019-12-09T09:12:20.123 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.0] 2019-12-09T09:12:20.123 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDmD+5dmvL2NBAAsl0iUdXFWiEBVROZTrxaZA== 2019-12-09T09:12:20.123 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.1] 2019-12-09T09:12:20.123 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDnD+5dKEcgEBAAUj1JX2sDbtEsRjKtkfj6FA== 2019-12-09T09:12:20.124 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.2] 2019-12-09T09:12:20.124 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDnD+5dohPPJhAARp3Ufc7efBN3fhvYPJqQZA== 2019-12-09T09:12:20.124 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.3] 2019-12-09T09:12:20.124 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDoD+5drt8FAhAAUYMtGTGBty1xCy1cOpfAyw== 2019-12-09T09:12:20.124 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.x] 2019-12-09T09:12:20.124 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDiD+5dxxNNHhAAfoecEUwFmZlwqxPSi6CGgA== 2019-12-09T09:12:20.125 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.4] 2019-12-09T09:12:20.125 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDsD+5dEYhQEhAA5mIZF98FJaH8HgAnLpjy7A== 2019-12-09T09:12:20.125 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.5] 2019-12-09T09:12:20.125 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDsD+5dMmRqNhAAn9FbVIQ4HFjk7gLlHV+Wtw== 2019-12-09T09:12:20.125 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.6] 2019-12-09T09:12:20.125 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDtD+5dhqJKGhAA5cTQQyJWnKlo3QdZWuQbRg== 2019-12-09T09:12:20.125 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.7] 2019-12-09T09:12:20.126 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDtD+5dI86QMxAAzGzpRJsPOsOTUy3uKSJQOQ== 2019-12-09T09:12:20.126 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2019-12-09T09:12:20.126 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDiD+5dah+/JhAAyzoaElvoIcaTYCeDWeYfMg== 2019-12-09T09:12:20.126 INFO:teuthology.orchestra.run.smithi022.stdout:[client.1] 2019-12-09T09:12:20.126 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDiD+5d+u5uMxAAkqhf7gS2KeEJdct1R3i6kA== 2019-12-09T09:12:20.136 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr.y] 2019-12-09T09:12:20.136 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDiD+5dFND0EBAAiD4RLNXaeSeodoG95QywcQ== 2019-12-09T09:12:20.136 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.0] 2019-12-09T09:12:20.136 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDmD+5dmvL2NBAAsl0iUdXFWiEBVROZTrxaZA== 2019-12-09T09:12:20.137 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.1] 2019-12-09T09:12:20.137 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDnD+5dKEcgEBAAUj1JX2sDbtEsRjKtkfj6FA== 2019-12-09T09:12:20.137 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.2] 2019-12-09T09:12:20.137 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDnD+5dohPPJhAARp3Ufc7efBN3fhvYPJqQZA== 2019-12-09T09:12:20.137 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.3] 2019-12-09T09:12:20.137 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDoD+5drt8FAhAAUYMtGTGBty1xCy1cOpfAyw== 2019-12-09T09:12:20.137 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr.x] 2019-12-09T09:12:20.138 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDiD+5dxxNNHhAAfoecEUwFmZlwqxPSi6CGgA== 2019-12-09T09:12:20.138 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.4] 2019-12-09T09:12:20.138 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDsD+5dEYhQEhAA5mIZF98FJaH8HgAnLpjy7A== 2019-12-09T09:12:20.138 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.5] 2019-12-09T09:12:20.138 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDsD+5dMmRqNhAAn9FbVIQ4HFjk7gLlHV+Wtw== 2019-12-09T09:12:20.138 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.6] 2019-12-09T09:12:20.138 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDtD+5dhqJKGhAA5cTQQyJWnKlo3QdZWuQbRg== 2019-12-09T09:12:20.138 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.7] 2019-12-09T09:12:20.139 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDtD+5dI86QMxAAzGzpRJsPOsOTUy3uKSJQOQ== 2019-12-09T09:12:20.139 INFO:teuthology.orchestra.run.smithi035.stdout:[client.0] 2019-12-09T09:12:20.139 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDiD+5dah+/JhAAyzoaElvoIcaTYCeDWeYfMg== 2019-12-09T09:12:20.139 INFO:teuthology.orchestra.run.smithi035.stdout:[client.1] 2019-12-09T09:12:20.139 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDiD+5d+u5uMxAAkqhf7gS2KeEJdct1R3i6kA== 2019-12-09T09:12:20.140 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:12:20.151 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:12:20.316 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:20.319 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:20.485 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:20.488 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:20.647 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:20.650 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:20.819 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:20.824 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:20.993 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:12:20.996 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:12:21.158 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:21.162 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:21.321 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:21.324 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:21.497 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:21.500 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:21.670 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:21.674 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:12:21.849 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:12:21.852 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:12:22.031 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:12:22.034 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:12:22.215 INFO:tasks.ceph:Running mkfs on mon nodes... 2019-12-09T09:12:22.215 INFO:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2019-12-09T09:12:22.336 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T09:12:22.520 INFO:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2019-12-09T09:12:22.642 INFO:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2019-12-09T09:12:22.743 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T09:12:22.900 INFO:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2019-12-09T09:12:22.989 INFO:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2019-12-09T09:12:23.113 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T09:12:23.285 INFO:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2019-12-09T09:12:23.397 INFO:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:23.401 INFO:teuthology.orchestra.run.smithi035:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:12:23.490 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2019-12-09T09:12:23.490 DEBUG:teuthology.misc:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T09:12:23.491 INFO:tasks.ceph.mon.a:Restarting daemon 2019-12-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2019-12-09T09:12:23.494 INFO:tasks.ceph.mon.a:Started 2019-12-09T09:12:23.494 DEBUG:teuthology.misc:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T09:12:23.495 INFO:tasks.ceph.mon.c:Restarting daemon 2019-12-09T09:12:23.495 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2019-12-09T09:12:23.498 INFO:tasks.ceph.mon.c:Started 2019-12-09T09:12:23.498 DEBUG:teuthology.misc:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T09:12:23.498 INFO:tasks.ceph.mon.b:Restarting daemon 2019-12-09T09:12:23.499 INFO:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2019-12-09T09:12:23.503 INFO:tasks.ceph.mon.b:Started 2019-12-09T09:12:23.503 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2019-12-09T09:12:23.503 INFO:tasks.ceph.mgr.y:Restarting daemon 2019-12-09T09:12:23.503 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2019-12-09T09:12:23.506 INFO:tasks.ceph.mgr.y:Started 2019-12-09T09:12:23.507 INFO:tasks.ceph.mgr.x:Restarting daemon 2019-12-09T09:12:23.507 INFO:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2019-12-09T09:12:23.510 INFO:tasks.ceph.mgr.x:Started 2019-12-09T09:12:23.510 INFO:tasks.ceph:Setting crush tunables to default 2019-12-09T09:12:23.510 INFO:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd crush tunables default 2019-12-09T09:12:27.738 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:00:04.050 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:12:27.838 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:00:04.105 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:12:27.855 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:00:04.167 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:12:41.097 INFO:teuthology.orchestra.run.smithi035.stderr:adjusted tunables profile to default 2019-12-09T09:12:41.112 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2019-12-09T09:12:41.112 INFO:teuthology.orchestra.run.smithi035:> mktemp 2019-12-09T09:12:41.161 INFO:teuthology.orchestra.run.smithi035.stdout:/tmp/tmp.FNGEZjja9b 2019-12-09T09:12:41.161 INFO:teuthology.orchestra.run.smithi035:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.FNGEZjja9b 2019-12-09T09:12:41.224 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 0666 /tmp/tmp.FNGEZjja9b 2019-12-09T09:12:41.326 DEBUG:teuthology.orchestra.remote:smithi035:/tmp/tmp.FNGEZjja9b is 37B 2019-12-09T09:12:41.373 INFO:teuthology.orchestra.run.smithi035:> rm -fr /tmp/tmp.FNGEZjja9b 2019-12-09T09:12:41.416 INFO:teuthology.orchestra.run.smithi035:> mktemp 2019-12-09T09:12:41.497 INFO:teuthology.orchestra.run.smithi035.stdout:/tmp/tmp.fbPxq5apwy 2019-12-09T09:12:41.498 INFO:teuthology.orchestra.run.smithi035:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.fbPxq5apwy 2019-12-09T09:12:41.635 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 0666 /tmp/tmp.fbPxq5apwy 2019-12-09T09:12:41.761 DEBUG:teuthology.orchestra.remote:smithi035:/tmp/tmp.fbPxq5apwy is 37B 2019-12-09T09:12:41.818 INFO:teuthology.orchestra.run.smithi035:> rm -fr /tmp/tmp.fbPxq5apwy 2019-12-09T09:12:41.862 INFO:teuthology.orchestra.run.smithi035:> mktemp 2019-12-09T09:12:41.944 INFO:teuthology.orchestra.run.smithi035.stdout:/tmp/tmp.rtr5XMtulK 2019-12-09T09:12:41.944 INFO:teuthology.orchestra.run.smithi035:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.rtr5XMtulK 2019-12-09T09:12:42.001 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 0666 /tmp/tmp.rtr5XMtulK 2019-12-09T09:12:42.100 DEBUG:teuthology.orchestra.remote:smithi035:/tmp/tmp.rtr5XMtulK is 37B 2019-12-09T09:12:42.151 INFO:teuthology.orchestra.run.smithi035:> rm -fr /tmp/tmp.rtr5XMtulK 2019-12-09T09:12:42.193 INFO:teuthology.orchestra.run.smithi035:> mktemp 2019-12-09T09:12:42.280 INFO:teuthology.orchestra.run.smithi035.stdout:/tmp/tmp.hIPzZ3HT4R 2019-12-09T09:12:42.280 INFO:teuthology.orchestra.run.smithi035:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.hIPzZ3HT4R 2019-12-09T09:12:42.336 INFO:teuthology.orchestra.run.smithi035:> sudo chmod 0666 /tmp/tmp.hIPzZ3HT4R 2019-12-09T09:12:42.450 DEBUG:teuthology.orchestra.remote:smithi035:/tmp/tmp.hIPzZ3HT4R is 37B 2019-12-09T09:12:42.501 INFO:teuthology.orchestra.run.smithi035:> rm -fr /tmp/tmp.hIPzZ3HT4R 2019-12-09T09:12:42.549 INFO:teuthology.orchestra.run.smithi022:> mktemp 2019-12-09T09:12:42.634 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.NWwaAnMQNK 2019-12-09T09:12:42.635 INFO:teuthology.orchestra.run.smithi022:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.NWwaAnMQNK 2019-12-09T09:12:42.724 INFO:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.NWwaAnMQNK 2019-12-09T09:12:42.841 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.NWwaAnMQNK is 37B 2019-12-09T09:12:42.893 INFO:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.NWwaAnMQNK 2019-12-09T09:12:42.940 INFO:teuthology.orchestra.run.smithi022:> mktemp 2019-12-09T09:12:43.036 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.fQmwiL7xtx 2019-12-09T09:12:43.039 INFO:teuthology.orchestra.run.smithi022:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.fQmwiL7xtx 2019-12-09T09:12:43.104 INFO:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.fQmwiL7xtx 2019-12-09T09:12:43.217 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.fQmwiL7xtx is 37B 2019-12-09T09:12:43.275 INFO:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.fQmwiL7xtx 2019-12-09T09:12:43.317 INFO:teuthology.orchestra.run.smithi022:> mktemp 2019-12-09T09:12:43.421 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.01mFnol6Ja 2019-12-09T09:12:43.422 INFO:teuthology.orchestra.run.smithi022:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.01mFnol6Ja 2019-12-09T09:12:43.499 INFO:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.01mFnol6Ja 2019-12-09T09:12:43.624 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.01mFnol6Ja is 37B 2019-12-09T09:12:43.677 INFO:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.01mFnol6Ja 2019-12-09T09:12:43.734 INFO:teuthology.orchestra.run.smithi022:> mktemp 2019-12-09T09:12:43.779 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.Yf8ozk52Ay 2019-12-09T09:12:43.780 INFO:teuthology.orchestra.run.smithi022:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.Yf8ozk52Ay 2019-12-09T09:12:43.899 INFO:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.Yf8ozk52Ay 2019-12-09T09:12:44.023 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.Yf8ozk52Ay is 37B 2019-12-09T09:12:44.072 INFO:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.Yf8ozk52Ay 2019-12-09T09:12:44.116 INFO:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd new 882655cd-a8d6-4dbc-b7f1-7e4396cf89b3 0 2019-12-09T09:12:45.399 INFO:teuthology.orchestra.run.smithi022.stdout:0 2019-12-09T09:12:45.422 INFO:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd new 624d58e2-6941-4a4d-b174-2c411795e304 1 2019-12-09T09:12:46.588 INFO:teuthology.orchestra.run.smithi022.stdout:1 2019-12-09T09:12:46.612 INFO:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd new e5801a97-b227-4c41-b6f2-3b56bb1a8cc6 2 2019-12-09T09:12:47.738 INFO:teuthology.orchestra.run.smithi022.stdout:2 2019-12-09T09:12:47.763 INFO:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd new db5d6690-575e-49d5-b237-55d43ad95fa1 3 2019-12-09T09:12:48.745 INFO:teuthology.orchestra.run.smithi022.stdout:3 2019-12-09T09:12:48.771 INFO:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd new 935b4864-b72e-4f29-b132-fee05fefc1d1 4 2019-12-09T09:12:50.045 INFO:teuthology.orchestra.run.smithi022.stdout:4 2019-12-09T09:12:50.067 INFO:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd new 8db0d933-b38c-49d3-bb0d-db45065fe4ad 5 2019-12-09T09:12:51.282 INFO:teuthology.orchestra.run.smithi022.stdout:5 2019-12-09T09:12:51.305 INFO:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd new ee89a743-a9da-45a5-b981-96403f6a7332 6 2019-12-09T09:12:52.455 INFO:teuthology.orchestra.run.smithi022.stdout:6 2019-12-09T09:12:52.476 INFO:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd new 55d3d417-5e49-4e54-ab11-d8c0b4681f99 7 2019-12-09T09:12:53.511 INFO:teuthology.orchestra.run.smithi022.stdout:7 2019-12-09T09:12:53.528 DEBUG:teuthology.misc:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:12:53.528 INFO:tasks.ceph.osd.0:Restarting daemon 2019-12-09T09:12:53.529 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 0 2019-12-09T09:12:53.533 INFO:tasks.ceph.osd.0:Started 2019-12-09T09:12:53.533 DEBUG:teuthology.misc:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:12:53.533 INFO:tasks.ceph.osd.1:Restarting daemon 2019-12-09T09:12:53.534 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 1 2019-12-09T09:12:53.537 INFO:tasks.ceph.osd.1:Started 2019-12-09T09:12:53.537 DEBUG:teuthology.misc:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:12:53.538 INFO:tasks.ceph.osd.2:Restarting daemon 2019-12-09T09:12:53.538 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 2 2019-12-09T09:12:53.541 INFO:tasks.ceph.osd.2:Started 2019-12-09T09:12:53.541 DEBUG:teuthology.misc:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:12:53.541 INFO:tasks.ceph.osd.3:Restarting daemon 2019-12-09T09:12:53.542 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 3 2019-12-09T09:12:53.545 INFO:tasks.ceph.osd.3:Started 2019-12-09T09:12:53.545 DEBUG:teuthology.misc:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:12:53.545 INFO:tasks.ceph.osd.4:Restarting daemon 2019-12-09T09:12:53.546 INFO:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 4 2019-12-09T09:12:53.549 INFO:tasks.ceph.osd.4:Started 2019-12-09T09:12:53.549 DEBUG:teuthology.misc:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:12:53.549 INFO:tasks.ceph.osd.5:Restarting daemon 2019-12-09T09:12:53.550 INFO:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 5 2019-12-09T09:12:53.553 INFO:tasks.ceph.osd.5:Started 2019-12-09T09:12:53.553 DEBUG:teuthology.misc:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:12:53.553 INFO:tasks.ceph.osd.6:Restarting daemon 2019-12-09T09:12:53.555 INFO:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 6 2019-12-09T09:12:53.561 INFO:tasks.ceph.osd.6:Started 2019-12-09T09:12:53.561 DEBUG:teuthology.misc:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:12:53.561 INFO:tasks.ceph.osd.7:Restarting daemon 2019-12-09T09:12:53.562 INFO:teuthology.orchestra.run.smithi022:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 7 2019-12-09T09:12:53.565 INFO:tasks.ceph.osd.7:Started 2019-12-09T09:12:53.565 INFO:tasks.ceph:Waiting for OSDs to come up 2019-12-09T09:12:54.387 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T09:12:55.050 INFO:teuthology.misc.health.smithi035.stdout: 2019-12-09T09:12:55.050 INFO:teuthology.misc.health.smithi035.stdout:{"epoch":10,"fsid":"92300541-fe50-4dfa-a6fe-782b29b49e98","created":"2019-12-09T09:12:37.233139+0000","modified":"2019-12-09T09:12:53.360386+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"882655cd-a8d6-4dbc-b7f1-7e4396cf89b3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"624d58e2-6941-4a4d-b174-2c411795e304","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e5801a97-b227-4c41-b6f2-3b56bb1a8cc6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"db5d6690-575e-49d5-b237-55d43ad95fa1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"935b4864-b72e-4f29-b132-fee05fefc1d1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8db0d933-b38c-49d3-bb0d-db45065fe4ad","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ee89a743-a9da-45a5-b981-96403f6a7332","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"55d3d417-5e49-4e54-ab11-d8c0b4681f99","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:12:55.061 DEBUG:teuthology.misc:0 of 8 OSDs are up 2019-12-09T09:12:57.444 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T09:12:57.434+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:12:57.493 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T09:12:57.483+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:12:57.527 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T09:12:57.512+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:12:57.553 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T09:12:57.512+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:12:57.554 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T09:12:57.530+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:12:57.642 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T09:12:57.608+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:12:58.380 INFO:tasks.ceph.osd.7.smithi022.stderr:==00:00:00:04.695 9813== Warning: unimplemented fcntl command: 1036 2019-12-09T09:12:58.448 INFO:tasks.ceph.osd.6.smithi022.stderr:==00:00:00:04.747 9815== Warning: unimplemented fcntl command: 1036 2019-12-09T09:12:58.465 INFO:tasks.ceph.osd.4.smithi022.stderr:==00:00:00:04.788 9812== Warning: unimplemented fcntl command: 1036 2019-12-09T09:12:58.470 INFO:tasks.ceph.osd.5.smithi022.stderr:==00:00:00:04.779 9814== Warning: unimplemented fcntl command: 1036 2019-12-09T09:12:58.521 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T09:12:58.482+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:12:58.601 INFO:tasks.ceph.osd.1.smithi035.stderr:==00:00:00:04.914 9790== Warning: unimplemented fcntl command: 1036 2019-12-09T09:12:58.601 INFO:tasks.ceph.osd.0.smithi035.stderr:==00:00:00:04.919 9789== Warning: unimplemented fcntl command: 1036 2019-12-09T09:12:58.717 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T09:12:58.685+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:12:58.755 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T09:12:58.742+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:58.763 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T09:12:58.758+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:58.785 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T09:12:58.761+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:58.846 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T09:12:58.832+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:58.882 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T09:12:58.849+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:58.895 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T09:12:58.862+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:58.903 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T09:12:58.878+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:58.904 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T09:12:58.886+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:58.905 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T09:12:58.854+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:58.927 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T09:12:58.900+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:58.934 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T09:12:58.889+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:58.943 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T09:12:58.903+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:58.988 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T09:12:58.940+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:59.023 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T09:12:58.956+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:59.025 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T09:12:58.959+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:59.046 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T09:12:59.012+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:12:59.063 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T09:12:59.029+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:59.085 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T09:12:59.033+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:12:59.401 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T09:12:59.398+0000 9b13800 -1 osd.7 0 log_to_monitors {default=true} 2019-12-09T09:12:59.468 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T09:12:59.467+0000 9b13800 -1 osd.5 0 log_to_monitors {default=true} 2019-12-09T09:12:59.504 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T09:12:59.503+0000 9b13800 -1 osd.4 0 log_to_monitors {default=true} 2019-12-09T09:12:59.520 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T09:12:59.518+0000 9b13800 -1 osd.6 0 log_to_monitors {default=true} 2019-12-09T09:12:59.541 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T09:12:59.519+0000 9b13800 -1 osd.0 0 log_to_monitors {default=true} 2019-12-09T09:12:59.571 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T09:12:59.550+0000 9b13800 -1 osd.1 0 log_to_monitors {default=true} 2019-12-09T09:12:59.727 INFO:tasks.ceph.osd.2.smithi035.stderr:==00:00:00:06.038 9792== Warning: unimplemented fcntl command: 1036 2019-12-09T09:13:00.020 INFO:tasks.ceph.osd.3.smithi035.stderr:==00:00:00:06.333 9791== Warning: unimplemented fcntl command: 1036 2019-12-09T09:13:00.266 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T09:13:00.185+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:13:00.319 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T09:13:00.236+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:13:00.348 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T09:13:00.248+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:13:00.531 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T09:13:00.507+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:13:00.550 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T09:13:00.525+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:13:00.584 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T09:13:00.529+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:13:00.942 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T09:13:00.918+0000 9b13800 -1 osd.2 0 log_to_monitors {default=true} 2019-12-09T09:13:01.194 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T09:13:01.173+0000 9b13800 -1 osd.3 0 log_to_monitors {default=true} 2019-12-09T09:13:01.808 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T09:13:01.780+0000 29a5f700 -1 osd.1 0 waiting for initial osdmap 2019-12-09T09:13:01.822 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T09:13:01.815+0000 29a5f700 -1 osd.4 0 waiting for initial osdmap 2019-12-09T09:13:01.844 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T09:13:01.832+0000 29a5f700 -1 osd.7 0 waiting for initial osdmap 2019-12-09T09:13:01.856 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T09:13:01.848+0000 29a5f700 -1 osd.5 0 waiting for initial osdmap 2019-12-09T09:13:01.867 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T09:13:01.907 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T09:13:01.900+0000 29a5f700 -1 osd.6 0 waiting for initial osdmap 2019-12-09T09:13:01.923 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T09:13:01.892+0000 29a5f700 -1 osd.0 0 waiting for initial osdmap 2019-12-09T09:13:02.793 INFO:teuthology.misc.health.smithi035.stdout: 2019-12-09T09:13:02.794 INFO:teuthology.misc.health.smithi035.stdout:{"epoch":13,"fsid":"92300541-fe50-4dfa-a6fe-782b29b49e98","created":"2019-12-09T09:12:37.233139+0000","modified":"2019-12-09T09:13:02.394354+0000","last_up_change":"2019-12-09T09:13:02.394354+0000","last_in_change":"2019-12-09T09:13:02.394354+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"882655cd-a8d6-4dbc-b7f1-7e4396cf89b3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"624d58e2-6941-4a4d-b174-2c411795e304","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e5801a97-b227-4c41-b6f2-3b56bb1a8cc6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"db5d6690-575e-49d5-b237-55d43ad95fa1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"935b4864-b72e-4f29-b132-fee05fefc1d1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8db0d933-b38c-49d3-bb0d-db45065fe4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":9814},{"type":"v1","addr":"172.21.15.22:6811","nonce":9814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":9814},{"type":"v1","addr":"172.21.15.22:6813","nonce":9814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":9814},{"type":"v1","addr":"172.21.15.22:6817","nonce":9814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":9814},{"type":"v1","addr":"172.21.15.22:6815","nonce":9814}]},"public_addr":"172.21.15.22:6811/9814","cluster_addr":"172.21.15.22:6813/9814","heartbeat_back_addr":"172.21.15.22:6817/9814","heartbeat_front_addr":"172.21.15.22:6815/9814","state":["exists","up"]},{"osd":6,"uuid":"ee89a743-a9da-45a5-b981-96403f6a7332","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"55d3d417-5e49-4e54-ab11-d8c0b4681f99","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:13:02.801 DEBUG:teuthology.misc:1 of 8 OSDs are up 2019-12-09T09:13:03.001 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T09:13:02.972+0000 29a5f700 -1 osd.2 0 waiting for initial osdmap 2019-12-09T09:13:04.086 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T09:13:04.061+0000 29a5f700 -1 osd.3 0 waiting for initial osdmap 2019-12-09T09:13:09.609 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T09:13:10.479 INFO:teuthology.misc.health.smithi035.stdout: 2019-12-09T09:13:10.480 INFO:teuthology.misc.health.smithi035.stdout:{"epoch":16,"fsid":"92300541-fe50-4dfa-a6fe-782b29b49e98","created":"2019-12-09T09:12:37.233139+0000","modified":"2019-12-09T09:13:05.693628+0000","last_up_change":"2019-12-09T09:13:04.638634+0000","last_in_change":"2019-12-09T09:13:04.638634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"882655cd-a8d6-4dbc-b7f1-7e4396cf89b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6808","nonce":9789},{"type":"v1","addr":"172.21.15.35:6809","nonce":9789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6810","nonce":9789},{"type":"v1","addr":"172.21.15.35:6811","nonce":9789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6814","nonce":9789},{"type":"v1","addr":"172.21.15.35:6815","nonce":9789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6812","nonce":9789},{"type":"v1","addr":"172.21.15.35:6813","nonce":9789}]},"public_addr":"172.21.15.35:6809/9789","cluster_addr":"172.21.15.35:6811/9789","heartbeat_back_addr":"172.21.15.35:6815/9789","heartbeat_front_addr":"172.21.15.35:6813/9789","state":["exists","up"]},{"osd":1,"uuid":"624d58e2-6941-4a4d-b174-2c411795e304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":9790},{"type":"v1","addr":"172.21.15.35:6801","nonce":9790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":9790},{"type":"v1","addr":"172.21.15.35:6803","nonce":9790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":9790},{"type":"v1","addr":"172.21.15.35:6807","nonce":9790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":9790},{"type":"v1","addr":"172.21.15.35:6805","nonce":9790}]},"public_addr":"172.21.15.35:6801/9790","cluster_addr":"172.21.15.35:6803/9790","heartbeat_back_addr":"172.21.15.35:6807/9790","heartbeat_front_addr":"172.21.15.35:6805/9790","state":["exists","up"]},{"osd":2,"uuid":"e5801a97-b227-4c41-b6f2-3b56bb1a8cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6816","nonce":9792},{"type":"v1","addr":"172.21.15.35:6817","nonce":9792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6818","nonce":9792},{"type":"v1","addr":"172.21.15.35:6819","nonce":9792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6822","nonce":9792},{"type":"v1","addr":"172.21.15.35:6823","nonce":9792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6820","nonce":9792},{"type":"v1","addr":"172.21.15.35:6821","nonce":9792}]},"public_addr":"172.21.15.35:6817/9792","cluster_addr":"172.21.15.35:6819/9792","heartbeat_back_addr":"172.21.15.35:6823/9792","heartbeat_front_addr":"172.21.15.35:6821/9792","state":["exists","up"]},{"osd":3,"uuid":"db5d6690-575e-49d5-b237-55d43ad95fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6824","nonce":9791},{"type":"v1","addr":"172.21.15.35:6825","nonce":9791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6826","nonce":9791},{"type":"v1","addr":"172.21.15.35:6827","nonce":9791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6830","nonce":9791},{"type":"v1","addr":"172.21.15.35:6831","nonce":9791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6828","nonce":9791},{"type":"v1","addr":"172.21.15.35:6829","nonce":9791}]},"public_addr":"172.21.15.35:6825/9791","cluster_addr":"172.21.15.35:6827/9791","heartbeat_back_addr":"172.21.15.35:6831/9791","heartbeat_front_addr":"172.21.15.35:6829/9791","state":["exists","up"]},{"osd":4,"uuid":"935b4864-b72e-4f29-b132-fee05fefc1d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":9812},{"type":"v1","addr":"172.21.15.22:6824","nonce":9812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6827","nonce":9812},{"type":"v1","addr":"172.21.15.22:6829","nonce":9812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":9812},{"type":"v1","addr":"172.21.15.22:6833","nonce":9812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":9812},{"type":"v1","addr":"172.21.15.22:6831","nonce":9812}]},"public_addr":"172.21.15.22:6824/9812","cluster_addr":"172.21.15.22:6829/9812","heartbeat_back_addr":"172.21.15.22:6833/9812","heartbeat_front_addr":"172.21.15.22:6831/9812","state":["exists","up"]},{"osd":5,"uuid":"8db0d933-b38c-49d3-bb0d-db45065fe4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":9814},{"type":"v1","addr":"172.21.15.22:6811","nonce":9814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":9814},{"type":"v1","addr":"172.21.15.22:6813","nonce":9814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":9814},{"type":"v1","addr":"172.21.15.22:6817","nonce":9814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":9814},{"type":"v1","addr":"172.21.15.22:6815","nonce":9814}]},"public_addr":"172.21.15.22:6811/9814","cluster_addr":"172.21.15.22:6813/9814","heartbeat_back_addr":"172.21.15.22:6817/9814","heartbeat_front_addr":"172.21.15.22:6815/9814","state":["exists","up"]},{"osd":6,"uuid":"ee89a743-a9da-45a5-b981-96403f6a7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":9815},{"type":"v1","addr":"172.21.15.22:6819","nonce":9815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":9815},{"type":"v1","addr":"172.21.15.22:6821","nonce":9815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":9815},{"type":"v1","addr":"172.21.15.22:6828","nonce":9815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":9815},{"type":"v1","addr":"172.21.15.22:6825","nonce":9815}]},"public_addr":"172.21.15.22:6819/9815","cluster_addr":"172.21.15.22:6821/9815","heartbeat_back_addr":"172.21.15.22:6828/9815","heartbeat_front_addr":"172.21.15.22:6825/9815","state":["exists","up"]},{"osd":7,"uuid":"55d3d417-5e49-4e54-ab11-d8c0b4681f99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":9813},{"type":"v1","addr":"172.21.15.22:6803","nonce":9813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":9813},{"type":"v1","addr":"172.21.15.22:6805","nonce":9813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":9813},{"type":"v1","addr":"172.21.15.22:6809","nonce":9813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":9813},{"type":"v1","addr":"172.21.15.22:6807","nonce":9813}]},"public_addr":"172.21.15.22:6803/9813","cluster_addr":"172.21.15.22:6805/9813","heartbeat_back_addr":"172.21.15.22:6809/9813","heartbeat_front_addr":"172.21.15.22:6807/9813","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.599674+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.660932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:02.074961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:02.270528+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.595659+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.542058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.603403+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.511943+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:13:10.496 DEBUG:teuthology.misc:8 of 8 OSDs are up 2019-12-09T09:13:10.496 INFO:tasks.ceph:Creating RBD pool 2019-12-09T09:13:10.497 INFO:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd pool create rbd 8 2019-12-09T09:13:13.082 INFO:teuthology.orchestra.run.smithi035.stderr:pool 'rbd' created 2019-12-09T09:13:13.100 INFO:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2019-12-09T09:13:15.007 INFO:teuthology.orchestra.run.smithi035.stderr:enabled application 'rbd' on pool 'rbd' 2019-12-09T09:13:15.022 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2019-12-09T09:13:15.022 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T09:13:15.023 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2019-12-09T09:13:15.845 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:13:15.845 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":19,"fsid":"92300541-fe50-4dfa-a6fe-782b29b49e98","created":"2019-12-09T09:12:37.233139+0000","modified":"2019-12-09T09:13:14.745635+0000","last_up_change":"2019-12-09T09:13:04.638634+0000","last_in_change":"2019-12-09T09:13:04.638634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T09:13:11.552323+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"882655cd-a8d6-4dbc-b7f1-7e4396cf89b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6808","nonce":9789},{"type":"v1","addr":"172.21.15.35:6809","nonce":9789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6810","nonce":9789},{"type":"v1","addr":"172.21.15.35:6811","nonce":9789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6814","nonce":9789},{"type":"v1","addr":"172.21.15.35:6815","nonce":9789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6812","nonce":9789},{"type":"v1","addr":"172.21.15.35:6813","nonce":9789}]},"public_addr":"172.21.15.35:6809/9789","cluster_addr":"172.21.15.35:6811/9789","heartbeat_back_addr":"172.21.15.35:6815/9789","heartbeat_front_addr":"172.21.15.35:6813/9789","state":["exists","up"]},{"osd":1,"uuid":"624d58e2-6941-4a4d-b174-2c411795e304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":9790},{"type":"v1","addr":"172.21.15.35:6801","nonce":9790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":9790},{"type":"v1","addr":"172.21.15.35:6803","nonce":9790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":9790},{"type":"v1","addr":"172.21.15.35:6807","nonce":9790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":9790},{"type":"v1","addr":"172.21.15.35:6805","nonce":9790}]},"public_addr":"172.21.15.35:6801/9790","cluster_addr":"172.21.15.35:6803/9790","heartbeat_back_addr":"172.21.15.35:6807/9790","heartbeat_front_addr":"172.21.15.35:6805/9790","state":["exists","up"]},{"osd":2,"uuid":"e5801a97-b227-4c41-b6f2-3b56bb1a8cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6816","nonce":9792},{"type":"v1","addr":"172.21.15.35:6817","nonce":9792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6818","nonce":9792},{"type":"v1","addr":"172.21.15.35:6819","nonce":9792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6822","nonce":9792},{"type":"v1","addr":"172.21.15.35:6823","nonce":9792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6820","nonce":9792},{"type":"v1","addr":"172.21.15.35:6821","nonce":9792}]},"public_addr":"172.21.15.35:6817/9792","cluster_addr":"172.21.15.35:6819/9792","heartbeat_back_addr":"172.21.15.35:6823/9792","heartbeat_front_addr":"172.21.15.35:6821/9792","state":["exists","up"]},{"osd":3,"uuid":"db5d6690-575e-49d5-b237-55d43ad95fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6824","nonce":9791},{"type":"v1","addr":"172.21.15.35:6825","nonce":9791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6826","nonce":9791},{"type":"v1","addr":"172.21.15.35:6827","nonce":9791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6830","nonce":9791},{"type":"v1","addr":"172.21.15.35:6831","nonce":9791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6828","nonce":9791},{"type":"v1","addr":"172.21.15.35:6829","nonce":9791}]},"public_addr":"172.21.15.35:6825/9791","cluster_addr":"172.21.15.35:6827/9791","heartbeat_back_addr":"172.21.15.35:6831/9791","heartbeat_front_addr":"172.21.15.35:6829/9791","state":["exists","up"]},{"osd":4,"uuid":"935b4864-b72e-4f29-b132-fee05fefc1d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":9812},{"type":"v1","addr":"172.21.15.22:6824","nonce":9812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6827","nonce":9812},{"type":"v1","addr":"172.21.15.22:6829","nonce":9812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":9812},{"type":"v1","addr":"172.21.15.22:6833","nonce":9812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":9812},{"type":"v1","addr":"172.21.15.22:6831","nonce":9812}]},"public_addr":"172.21.15.22:6824/9812","cluster_addr":"172.21.15.22:6829/9812","heartbeat_back_addr":"172.21.15.22:6833/9812","heartbeat_front_addr":"172.21.15.22:6831/9812","state":["exists","up"]},{"osd":5,"uuid":"8db0d933-b38c-49d3-bb0d-db45065fe4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":9814},{"type":"v1","addr":"172.21.15.22:6811","nonce":9814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":9814},{"type":"v1","addr":"172.21.15.22:6813","nonce":9814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":9814},{"type":"v1","addr":"172.21.15.22:6817","nonce":9814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":9814},{"type":"v1","addr":"172.21.15.22:6815","nonce":9814}]},"public_addr":"172.21.15.22:6811/9814","cluster_addr":"172.21.15.22:6813/9814","heartbeat_back_addr":"172.21.15.22:6817/9814","heartbeat_front_addr":"172.21.15.22:6815/9814","state":["exists","up"]},{"osd":6,"uuid":"ee89a743-a9da-45a5-b981-96403f6a7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":9815},{"type":"v1","addr":"172.21.15.22:6819","nonce":9815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":9815},{"type":"v1","addr":"172.21.15.22:6821","nonce":9815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":9815},{"type":"v1","addr":"172.21.15.22:6828","nonce":9815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":9815},{"type":"v1","addr":"172.21.15.22:6825","nonce":9815}]},"public_addr":"172.21.15.22:6819/9815","cluster_addr":"172.21.15.22:6821/9815","heartbeat_back_addr":"172.21.15.22:6828/9815","heartbeat_front_addr":"172.21.15.22:6825/9815","state":["exists","up"]},{"osd":7,"uuid":"55d3d417-5e49-4e54-ab11-d8c0b4681f99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":9813},{"type":"v1","addr":"172.21.15.22:6803","nonce":9813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":9813},{"type":"v1","addr":"172.21.15.22:6805","nonce":9813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":9813},{"type":"v1","addr":"172.21.15.22:6809","nonce":9813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":9813},{"type":"v1","addr":"172.21.15.22:6807","nonce":9813}]},"public_addr":"172.21.15.22:6803/9813","cluster_addr":"172.21.15.22:6805/9813","heartbeat_back_addr":"172.21.15.22:6809/9813","heartbeat_front_addr":"172.21.15.22:6807/9813","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.599674+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.660932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:02.074961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:02.270528+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.595659+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.542058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.603403+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.511943+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:13:15.858 INFO:tasks.ceph.ceph_manager.ceph:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,creating', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2019-12-09T09:13:11.552323+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 32769, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'19', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2019-12-09T09:13:15.858 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2019-12-09T09:13:16.646 INFO:teuthology.orchestra.run.smithi035.stdout:pg_num: 8 2019-12-09T09:13:16.661 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2019-12-09T09:13:16.662 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2019-12-09T09:13:16.662 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2019-12-09T09:13:17.857 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:13:17.871 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":3,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":8484},{"type":"v1","addr":"172.21.15.22:6801","nonce":8484}]},"active_addr":"172.21.15.22:6801/8484","active_change":"2019-12-09T09:12:41.919049+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540138292836696063,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator_cli","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]}} 2019-12-09T09:13:17.878 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2019-12-09T09:13:17.878 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2019-12-09T09:13:17.878 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T09:13:19.021 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:13:19.021 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":20,"fsid":"92300541-fe50-4dfa-a6fe-782b29b49e98","created":"2019-12-09T09:12:37.233139+0000","modified":"2019-12-09T09:13:16.134825+0000","last_up_change":"2019-12-09T09:13:04.638634+0000","last_in_change":"2019-12-09T09:13:04.638634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T09:13:11.552323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"882655cd-a8d6-4dbc-b7f1-7e4396cf89b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6808","nonce":9789},{"type":"v1","addr":"172.21.15.35:6809","nonce":9789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6810","nonce":9789},{"type":"v1","addr":"172.21.15.35:6811","nonce":9789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6814","nonce":9789},{"type":"v1","addr":"172.21.15.35:6815","nonce":9789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6812","nonce":9789},{"type":"v1","addr":"172.21.15.35:6813","nonce":9789}]},"public_addr":"172.21.15.35:6809/9789","cluster_addr":"172.21.15.35:6811/9789","heartbeat_back_addr":"172.21.15.35:6815/9789","heartbeat_front_addr":"172.21.15.35:6813/9789","state":["exists","up"]},{"osd":1,"uuid":"624d58e2-6941-4a4d-b174-2c411795e304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":9790},{"type":"v1","addr":"172.21.15.35:6801","nonce":9790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":9790},{"type":"v1","addr":"172.21.15.35:6803","nonce":9790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":9790},{"type":"v1","addr":"172.21.15.35:6807","nonce":9790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":9790},{"type":"v1","addr":"172.21.15.35:6805","nonce":9790}]},"public_addr":"172.21.15.35:6801/9790","cluster_addr":"172.21.15.35:6803/9790","heartbeat_back_addr":"172.21.15.35:6807/9790","heartbeat_front_addr":"172.21.15.35:6805/9790","state":["exists","up"]},{"osd":2,"uuid":"e5801a97-b227-4c41-b6f2-3b56bb1a8cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6816","nonce":9792},{"type":"v1","addr":"172.21.15.35:6817","nonce":9792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6818","nonce":9792},{"type":"v1","addr":"172.21.15.35:6819","nonce":9792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6822","nonce":9792},{"type":"v1","addr":"172.21.15.35:6823","nonce":9792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6820","nonce":9792},{"type":"v1","addr":"172.21.15.35:6821","nonce":9792}]},"public_addr":"172.21.15.35:6817/9792","cluster_addr":"172.21.15.35:6819/9792","heartbeat_back_addr":"172.21.15.35:6823/9792","heartbeat_front_addr":"172.21.15.35:6821/9792","state":["exists","up"]},{"osd":3,"uuid":"db5d6690-575e-49d5-b237-55d43ad95fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6824","nonce":9791},{"type":"v1","addr":"172.21.15.35:6825","nonce":9791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6826","nonce":9791},{"type":"v1","addr":"172.21.15.35:6827","nonce":9791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6830","nonce":9791},{"type":"v1","addr":"172.21.15.35:6831","nonce":9791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6828","nonce":9791},{"type":"v1","addr":"172.21.15.35:6829","nonce":9791}]},"public_addr":"172.21.15.35:6825/9791","cluster_addr":"172.21.15.35:6827/9791","heartbeat_back_addr":"172.21.15.35:6831/9791","heartbeat_front_addr":"172.21.15.35:6829/9791","state":["exists","up"]},{"osd":4,"uuid":"935b4864-b72e-4f29-b132-fee05fefc1d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":9812},{"type":"v1","addr":"172.21.15.22:6824","nonce":9812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6827","nonce":9812},{"type":"v1","addr":"172.21.15.22:6829","nonce":9812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":9812},{"type":"v1","addr":"172.21.15.22:6833","nonce":9812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":9812},{"type":"v1","addr":"172.21.15.22:6831","nonce":9812}]},"public_addr":"172.21.15.22:6824/9812","cluster_addr":"172.21.15.22:6829/9812","heartbeat_back_addr":"172.21.15.22:6833/9812","heartbeat_front_addr":"172.21.15.22:6831/9812","state":["exists","up"]},{"osd":5,"uuid":"8db0d933-b38c-49d3-bb0d-db45065fe4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":9814},{"type":"v1","addr":"172.21.15.22:6811","nonce":9814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":9814},{"type":"v1","addr":"172.21.15.22:6813","nonce":9814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":9814},{"type":"v1","addr":"172.21.15.22:6817","nonce":9814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":9814},{"type":"v1","addr":"172.21.15.22:6815","nonce":9814}]},"public_addr":"172.21.15.22:6811/9814","cluster_addr":"172.21.15.22:6813/9814","heartbeat_back_addr":"172.21.15.22:6817/9814","heartbeat_front_addr":"172.21.15.22:6815/9814","state":["exists","up"]},{"osd":6,"uuid":"ee89a743-a9da-45a5-b981-96403f6a7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":9815},{"type":"v1","addr":"172.21.15.22:6819","nonce":9815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":9815},{"type":"v1","addr":"172.21.15.22:6821","nonce":9815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":9815},{"type":"v1","addr":"172.21.15.22:6828","nonce":9815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":9815},{"type":"v1","addr":"172.21.15.22:6825","nonce":9815}]},"public_addr":"172.21.15.22:6819/9815","cluster_addr":"172.21.15.22:6821/9815","heartbeat_back_addr":"172.21.15.22:6828/9815","heartbeat_front_addr":"172.21.15.22:6825/9815","state":["exists","up"]},{"osd":7,"uuid":"55d3d417-5e49-4e54-ab11-d8c0b4681f99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":9813},{"type":"v1","addr":"172.21.15.22:6803","nonce":9813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":9813},{"type":"v1","addr":"172.21.15.22:6805","nonce":9813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":9813},{"type":"v1","addr":"172.21.15.22:6809","nonce":9813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":9813},{"type":"v1","addr":"172.21.15.22:6807","nonce":9813}]},"public_addr":"172.21.15.22:6803/9813","cluster_addr":"172.21.15.22:6805/9813","heartbeat_back_addr":"172.21.15.22:6809/9813","heartbeat_front_addr":"172.21.15.22:6807/9813","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.599674+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.660932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:02.074961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:02.270528+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.595659+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.542058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.603403+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.511943+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:13:19.048 INFO:tasks.ceph.ceph_manager.ceph:all up! 2019-12-09T09:13:19.048 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T09:13:19.882 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:13:19.882 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":20,"fsid":"92300541-fe50-4dfa-a6fe-782b29b49e98","created":"2019-12-09T09:12:37.233139+0000","modified":"2019-12-09T09:13:16.134825+0000","last_up_change":"2019-12-09T09:13:04.638634+0000","last_in_change":"2019-12-09T09:13:04.638634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T09:13:11.552323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"882655cd-a8d6-4dbc-b7f1-7e4396cf89b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6808","nonce":9789},{"type":"v1","addr":"172.21.15.35:6809","nonce":9789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6810","nonce":9789},{"type":"v1","addr":"172.21.15.35:6811","nonce":9789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6814","nonce":9789},{"type":"v1","addr":"172.21.15.35:6815","nonce":9789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6812","nonce":9789},{"type":"v1","addr":"172.21.15.35:6813","nonce":9789}]},"public_addr":"172.21.15.35:6809/9789","cluster_addr":"172.21.15.35:6811/9789","heartbeat_back_addr":"172.21.15.35:6815/9789","heartbeat_front_addr":"172.21.15.35:6813/9789","state":["exists","up"]},{"osd":1,"uuid":"624d58e2-6941-4a4d-b174-2c411795e304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":9790},{"type":"v1","addr":"172.21.15.35:6801","nonce":9790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":9790},{"type":"v1","addr":"172.21.15.35:6803","nonce":9790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":9790},{"type":"v1","addr":"172.21.15.35:6807","nonce":9790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":9790},{"type":"v1","addr":"172.21.15.35:6805","nonce":9790}]},"public_addr":"172.21.15.35:6801/9790","cluster_addr":"172.21.15.35:6803/9790","heartbeat_back_addr":"172.21.15.35:6807/9790","heartbeat_front_addr":"172.21.15.35:6805/9790","state":["exists","up"]},{"osd":2,"uuid":"e5801a97-b227-4c41-b6f2-3b56bb1a8cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6816","nonce":9792},{"type":"v1","addr":"172.21.15.35:6817","nonce":9792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6818","nonce":9792},{"type":"v1","addr":"172.21.15.35:6819","nonce":9792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6822","nonce":9792},{"type":"v1","addr":"172.21.15.35:6823","nonce":9792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6820","nonce":9792},{"type":"v1","addr":"172.21.15.35:6821","nonce":9792}]},"public_addr":"172.21.15.35:6817/9792","cluster_addr":"172.21.15.35:6819/9792","heartbeat_back_addr":"172.21.15.35:6823/9792","heartbeat_front_addr":"172.21.15.35:6821/9792","state":["exists","up"]},{"osd":3,"uuid":"db5d6690-575e-49d5-b237-55d43ad95fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6824","nonce":9791},{"type":"v1","addr":"172.21.15.35:6825","nonce":9791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6826","nonce":9791},{"type":"v1","addr":"172.21.15.35:6827","nonce":9791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6830","nonce":9791},{"type":"v1","addr":"172.21.15.35:6831","nonce":9791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6828","nonce":9791},{"type":"v1","addr":"172.21.15.35:6829","nonce":9791}]},"public_addr":"172.21.15.35:6825/9791","cluster_addr":"172.21.15.35:6827/9791","heartbeat_back_addr":"172.21.15.35:6831/9791","heartbeat_front_addr":"172.21.15.35:6829/9791","state":["exists","up"]},{"osd":4,"uuid":"935b4864-b72e-4f29-b132-fee05fefc1d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":9812},{"type":"v1","addr":"172.21.15.22:6824","nonce":9812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6827","nonce":9812},{"type":"v1","addr":"172.21.15.22:6829","nonce":9812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":9812},{"type":"v1","addr":"172.21.15.22:6833","nonce":9812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":9812},{"type":"v1","addr":"172.21.15.22:6831","nonce":9812}]},"public_addr":"172.21.15.22:6824/9812","cluster_addr":"172.21.15.22:6829/9812","heartbeat_back_addr":"172.21.15.22:6833/9812","heartbeat_front_addr":"172.21.15.22:6831/9812","state":["exists","up"]},{"osd":5,"uuid":"8db0d933-b38c-49d3-bb0d-db45065fe4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":9814},{"type":"v1","addr":"172.21.15.22:6811","nonce":9814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":9814},{"type":"v1","addr":"172.21.15.22:6813","nonce":9814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":9814},{"type":"v1","addr":"172.21.15.22:6817","nonce":9814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":9814},{"type":"v1","addr":"172.21.15.22:6815","nonce":9814}]},"public_addr":"172.21.15.22:6811/9814","cluster_addr":"172.21.15.22:6813/9814","heartbeat_back_addr":"172.21.15.22:6817/9814","heartbeat_front_addr":"172.21.15.22:6815/9814","state":["exists","up"]},{"osd":6,"uuid":"ee89a743-a9da-45a5-b981-96403f6a7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":9815},{"type":"v1","addr":"172.21.15.22:6819","nonce":9815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":9815},{"type":"v1","addr":"172.21.15.22:6821","nonce":9815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":9815},{"type":"v1","addr":"172.21.15.22:6828","nonce":9815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6823","nonce":9815},{"type":"v1","addr":"172.21.15.22:6825","nonce":9815}]},"public_addr":"172.21.15.22:6819/9815","cluster_addr":"172.21.15.22:6821/9815","heartbeat_back_addr":"172.21.15.22:6828/9815","heartbeat_front_addr":"172.21.15.22:6825/9815","state":["exists","up"]},{"osd":7,"uuid":"55d3d417-5e49-4e54-ab11-d8c0b4681f99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":9813},{"type":"v1","addr":"172.21.15.22:6803","nonce":9813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":9813},{"type":"v1","addr":"172.21.15.22:6805","nonce":9813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":9813},{"type":"v1","addr":"172.21.15.22:6809","nonce":9813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":9813},{"type":"v1","addr":"172.21.15.22:6807","nonce":9813}]},"public_addr":"172.21.15.22:6803/9813","cluster_addr":"172.21.15.22:6805/9813","heartbeat_back_addr":"172.21.15.22:6809/9813","heartbeat_front_addr":"172.21.15.22:6807/9813","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.599674+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.660932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:02.074961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:02.270528+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.595659+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.542058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.603403+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:13:00.511943+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:13:19.902 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2019-12-09T09:13:20.478 INFO:teuthology.orchestra.run.smithi035.stdout:60129542149 2019-12-09T09:13:20.478 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2019-12-09T09:13:21.035 INFO:teuthology.orchestra.run.smithi035.stdout:60129542151 2019-12-09T09:13:21.035 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2019-12-09T09:13:21.561 INFO:teuthology.orchestra.run.smithi035.stdout:64424509446 2019-12-09T09:13:21.562 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2019-12-09T09:13:22.185 INFO:teuthology.orchestra.run.smithi035.stdout:64424509445 2019-12-09T09:13:22.185 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2019-12-09T09:13:22.800 INFO:teuthology.orchestra.run.smithi035.stdout:60129542149 2019-12-09T09:13:22.801 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2019-12-09T09:13:23.475 INFO:teuthology.orchestra.run.smithi035.stdout:55834574854 2019-12-09T09:13:23.497 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2019-12-09T09:13:24.198 INFO:teuthology.orchestra.run.smithi035.stdout:60129542150 2019-12-09T09:13:24.198 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2019-12-09T09:13:24.868 INFO:teuthology.orchestra.run.smithi035.stdout:60129542152 2019-12-09T09:13:24.869 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2019-12-09T09:13:25.806 INFO:teuthology.orchestra.run.smithi035.stdout:60129542150 2019-12-09T09:13:25.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.0 2019-12-09T09:13:25.820 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2019-12-09T09:13:26.721 INFO:teuthology.orchestra.run.smithi035.stdout:60129542152 2019-12-09T09:13:26.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542152 for osd.1 2019-12-09T09:13:26.741 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2019-12-09T09:13:27.583 INFO:teuthology.orchestra.run.smithi035.stdout:64424509447 2019-12-09T09:13:27.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.2 2019-12-09T09:13:27.597 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2019-12-09T09:13:28.356 INFO:teuthology.orchestra.run.smithi035.stdout:64424509446 2019-12-09T09:13:28.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.3 2019-12-09T09:13:28.370 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2019-12-09T09:13:29.209 INFO:teuthology.orchestra.run.smithi035.stdout:60129542150 2019-12-09T09:13:29.224 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.4 2019-12-09T09:13:29.224 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2019-12-09T09:13:30.058 INFO:teuthology.orchestra.run.smithi035.stdout:55834574855 2019-12-09T09:13:30.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.5 2019-12-09T09:13:30.077 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2019-12-09T09:13:30.946 INFO:teuthology.orchestra.run.smithi035.stdout:60129542151 2019-12-09T09:13:30.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.6 2019-12-09T09:13:30.960 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2019-12-09T09:13:31.895 INFO:teuthology.orchestra.run.smithi035.stdout:60129542153 2019-12-09T09:13:31.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542153 for osd.7 2019-12-09T09:13:31.909 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2019-12-09T09:13:31.909 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-12-09T09:13:32.786 INFO:teuthology.orchestra.run.smithi035.stderr:dumped all 2019-12-09T09:13:32.787 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:13:32.801 INFO:teuthology.orchestra.run.smithi035.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2019-12-09T09:13:32.188073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":1699648,"kb_used_data":747895392,"kb_used_omap":295,"kb_used_meta":0,"kb_avail":747895392,"statfs":{"total":767585320960,"available":765844881408,"internally_reserved":0,"allocated":765844881408,"data_stored":765844881408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":302099,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.354388+0000","last_change":"2019-12-09T09:13:15.333711+0000","last_active":"2019-12-09T09:13:15.354388+0000","last_peered":"2019-12-09T09:13:15.354388+0000","last_clean":"2019-12-09T09:13:15.354388+0000","last_became_active":"2019-12-09T09:13:15.297476+0000","last_became_peered":"2019-12-09T09:13:15.297476+0000","last_unstale":"2019-12-09T09:13:15.354388+0000","last_undegraded":"2019-12-09T09:13:15.354388+0000","last_fullsized":"2019-12-09T09:13:15.354388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.741390+0000","last_change":"2019-12-09T09:13:15.723043+0000","last_active":"2019-12-09T09:13:15.741390+0000","last_peered":"2019-12-09T09:13:15.741390+0000","last_clean":"2019-12-09T09:13:15.741390+0000","last_became_active":"2019-12-09T09:13:15.686119+0000","last_became_peered":"2019-12-09T09:13:15.686119+0000","last_unstale":"2019-12-09T09:13:15.741390+0000","last_undegraded":"2019-12-09T09:13:15.741390+0000","last_fullsized":"2019-12-09T09:13:15.741390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.732045+0000","last_change":"2019-12-09T09:13:15.684163+0000","last_active":"2019-12-09T09:13:15.732045+0000","last_peered":"2019-12-09T09:13:15.732045+0000","last_clean":"2019-12-09T09:13:15.732045+0000","last_became_active":"2019-12-09T09:13:15.635575+0000","last_became_peered":"2019-12-09T09:13:15.635575+0000","last_unstale":"2019-12-09T09:13:15.732045+0000","last_undegraded":"2019-12-09T09:13:15.732045+0000","last_fullsized":"2019-12-09T09:13:15.732045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.408630+0000","last_change":"2019-12-09T09:13:15.339888+0000","last_active":"2019-12-09T09:13:15.408630+0000","last_peered":"2019-12-09T09:13:15.408630+0000","last_clean":"2019-12-09T09:13:15.408630+0000","last_became_active":"2019-12-09T09:13:15.299734+0000","last_became_peered":"2019-12-09T09:13:15.299734+0000","last_unstale":"2019-12-09T09:13:15.408630+0000","last_undegraded":"2019-12-09T09:13:15.408630+0000","last_fullsized":"2019-12-09T09:13:15.408630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.822545+0000","last_change":"2019-12-09T09:13:15.800305+0000","last_active":"2019-12-09T09:13:15.822545+0000","last_peered":"2019-12-09T09:13:15.822545+0000","last_clean":"2019-12-09T09:13:15.822545+0000","last_became_active":"2019-12-09T09:13:15.751576+0000","last_became_peered":"2019-12-09T09:13:15.751576+0000","last_unstale":"2019-12-09T09:13:15.822545+0000","last_undegraded":"2019-12-09T09:13:15.822545+0000","last_fullsized":"2019-12-09T09:13:15.822545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.774289+0000","last_change":"2019-12-09T09:13:15.753160+0000","last_active":"2019-12-09T09:13:15.774289+0000","last_peered":"2019-12-09T09:13:15.774289+0000","last_clean":"2019-12-09T09:13:15.774289+0000","last_became_active":"2019-12-09T09:13:15.696999+0000","last_became_peered":"2019-12-09T09:13:15.696999+0000","last_unstale":"2019-12-09T09:13:15.774289+0000","last_undegraded":"2019-12-09T09:13:15.774289+0000","last_fullsized":"2019-12-09T09:13:15.774289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.820302+0000","last_change":"2019-12-09T09:13:15.785377+0000","last_active":"2019-12-09T09:13:15.820302+0000","last_peered":"2019-12-09T09:13:15.820302+0000","last_clean":"2019-12-09T09:13:15.820302+0000","last_became_active":"2019-12-09T09:13:15.742438+0000","last_became_peered":"2019-12-09T09:13:15.742438+0000","last_unstale":"2019-12-09T09:13:15.820302+0000","last_undegraded":"2019-12-09T09:13:15.820302+0000","last_fullsized":"2019-12-09T09:13:15.820302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.204453+0000","last_change":"2019-12-09T09:13:14.440789+0000","last_active":"2019-12-09T09:13:15.204453+0000","last_peered":"2019-12-09T09:13:15.204453+0000","last_clean":"2019-12-09T09:13:15.204453+0000","last_became_active":"2019-12-09T09:13:14.392508+0000","last_became_peered":"2019-12-09T09:13:14.392508+0000","last_unstale":"2019-12-09T09:13:15.204453+0000","last_undegraded":"2019-12-09T09:13:15.204453+0000","last_fullsized":"2019-12-09T09:13:15.204453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37812,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.218}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.656}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.884}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.277}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":45.843}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.487}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.966}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32954,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.754}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.367}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.905}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.410}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.665}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.333}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":42.472}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37692,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.493}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.835}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.502}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.857}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.676}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.974}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.434}]}]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32891,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":7.297}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":8.136}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.610}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.148}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.180}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.238}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47371,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":9.091}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.154}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":7.968}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.215}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.988}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.537}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.305}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37787,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.697}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.540}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.604}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.878}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.266}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.069}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.485}]}]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":41,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42657,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.310}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.596}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.819}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.582}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.206}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":42.567}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.938}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32935,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":9.800}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.891}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.276}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.907}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.033}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.458}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.132}]}]}]}} 2019-12-09T09:13:32.803 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-12-09T09:13:33.591 INFO:teuthology.orchestra.run.smithi035.stderr:dumped all 2019-12-09T09:13:33.592 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:13:33.605 INFO:teuthology.orchestra.run.smithi035.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2019-12-09T09:13:32.188073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":1699648,"kb_used_data":747895392,"kb_used_omap":295,"kb_used_meta":0,"kb_avail":747895392,"statfs":{"total":767585320960,"available":765844881408,"internally_reserved":0,"allocated":765844881408,"data_stored":765844881408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":302099,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.354388+0000","last_change":"2019-12-09T09:13:15.333711+0000","last_active":"2019-12-09T09:13:15.354388+0000","last_peered":"2019-12-09T09:13:15.354388+0000","last_clean":"2019-12-09T09:13:15.354388+0000","last_became_active":"2019-12-09T09:13:15.297476+0000","last_became_peered":"2019-12-09T09:13:15.297476+0000","last_unstale":"2019-12-09T09:13:15.354388+0000","last_undegraded":"2019-12-09T09:13:15.354388+0000","last_fullsized":"2019-12-09T09:13:15.354388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.741390+0000","last_change":"2019-12-09T09:13:15.723043+0000","last_active":"2019-12-09T09:13:15.741390+0000","last_peered":"2019-12-09T09:13:15.741390+0000","last_clean":"2019-12-09T09:13:15.741390+0000","last_became_active":"2019-12-09T09:13:15.686119+0000","last_became_peered":"2019-12-09T09:13:15.686119+0000","last_unstale":"2019-12-09T09:13:15.741390+0000","last_undegraded":"2019-12-09T09:13:15.741390+0000","last_fullsized":"2019-12-09T09:13:15.741390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.732045+0000","last_change":"2019-12-09T09:13:15.684163+0000","last_active":"2019-12-09T09:13:15.732045+0000","last_peered":"2019-12-09T09:13:15.732045+0000","last_clean":"2019-12-09T09:13:15.732045+0000","last_became_active":"2019-12-09T09:13:15.635575+0000","last_became_peered":"2019-12-09T09:13:15.635575+0000","last_unstale":"2019-12-09T09:13:15.732045+0000","last_undegraded":"2019-12-09T09:13:15.732045+0000","last_fullsized":"2019-12-09T09:13:15.732045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.408630+0000","last_change":"2019-12-09T09:13:15.339888+0000","last_active":"2019-12-09T09:13:15.408630+0000","last_peered":"2019-12-09T09:13:15.408630+0000","last_clean":"2019-12-09T09:13:15.408630+0000","last_became_active":"2019-12-09T09:13:15.299734+0000","last_became_peered":"2019-12-09T09:13:15.299734+0000","last_unstale":"2019-12-09T09:13:15.408630+0000","last_undegraded":"2019-12-09T09:13:15.408630+0000","last_fullsized":"2019-12-09T09:13:15.408630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.822545+0000","last_change":"2019-12-09T09:13:15.800305+0000","last_active":"2019-12-09T09:13:15.822545+0000","last_peered":"2019-12-09T09:13:15.822545+0000","last_clean":"2019-12-09T09:13:15.822545+0000","last_became_active":"2019-12-09T09:13:15.751576+0000","last_became_peered":"2019-12-09T09:13:15.751576+0000","last_unstale":"2019-12-09T09:13:15.822545+0000","last_undegraded":"2019-12-09T09:13:15.822545+0000","last_fullsized":"2019-12-09T09:13:15.822545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.774289+0000","last_change":"2019-12-09T09:13:15.753160+0000","last_active":"2019-12-09T09:13:15.774289+0000","last_peered":"2019-12-09T09:13:15.774289+0000","last_clean":"2019-12-09T09:13:15.774289+0000","last_became_active":"2019-12-09T09:13:15.696999+0000","last_became_peered":"2019-12-09T09:13:15.696999+0000","last_unstale":"2019-12-09T09:13:15.774289+0000","last_undegraded":"2019-12-09T09:13:15.774289+0000","last_fullsized":"2019-12-09T09:13:15.774289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.820302+0000","last_change":"2019-12-09T09:13:15.785377+0000","last_active":"2019-12-09T09:13:15.820302+0000","last_peered":"2019-12-09T09:13:15.820302+0000","last_clean":"2019-12-09T09:13:15.820302+0000","last_became_active":"2019-12-09T09:13:15.742438+0000","last_became_peered":"2019-12-09T09:13:15.742438+0000","last_unstale":"2019-12-09T09:13:15.820302+0000","last_undegraded":"2019-12-09T09:13:15.820302+0000","last_fullsized":"2019-12-09T09:13:15.820302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T09:13:15.204453+0000","last_change":"2019-12-09T09:13:14.440789+0000","last_active":"2019-12-09T09:13:15.204453+0000","last_peered":"2019-12-09T09:13:15.204453+0000","last_clean":"2019-12-09T09:13:15.204453+0000","last_became_active":"2019-12-09T09:13:14.392508+0000","last_became_peered":"2019-12-09T09:13:14.392508+0000","last_unstale":"2019-12-09T09:13:15.204453+0000","last_undegraded":"2019-12-09T09:13:15.204453+0000","last_fullsized":"2019-12-09T09:13:15.204453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:13:12.396057+0000","last_clean_scrub_stamp":"2019-12-09T09:13:12.396057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37812,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.218}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.656}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.884}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.277}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":45.843}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.487}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.966}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32954,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.754}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.367}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.905}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.410}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.665}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.333}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":42.472}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37692,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.493}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.835}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.502}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.857}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.676}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.974}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.434}]}]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32891,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":7.297}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":8.136}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.610}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.148}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.180}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.238}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47371,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":9.091}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.154}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":7.968}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.215}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.988}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.537}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.305}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37787,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.697}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.540}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.604}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.878}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.266}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.069}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.485}]}]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":41,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42657,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.310}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.596}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.819}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.582}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.206}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":42.567}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.938}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32935,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":9.800}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.891}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.276}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.907}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.033}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.458}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.132}]}]}]}} 2019-12-09T09:13:33.607 INFO:tasks.ceph.ceph_manager.ceph:clean! 2019-12-09T09:13:33.607 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2019-12-09T09:13:33.607 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2019-12-09T09:13:33.607 INFO:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2019-12-09T09:13:34.390 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:13:34.404 INFO:teuthology.orchestra.run.smithi035.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2019-12-09T09:13:34.404 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2019-12-09T09:13:34.404 INFO:teuthology.run_tasks:Running task openssl_keys... 2019-12-09T09:13:34.417 INFO:teuthology.run_tasks:Running task rgw... 2019-12-09T09:13:34.435 DEBUG:tasks.rgw:config is {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2019-12-09T09:13:34.436 DEBUG:tasks.rgw:client list is ['client.0'] 2019-12-09T09:13:34.436 INFO:tasks.rgw:Creating data pools 2019-12-09T09:13:34.437 DEBUG:tasks.rgw:Obtaining remote for client client.0 2019-12-09T09:13:34.437 INFO:teuthology.orchestra.run.smithi035:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=2 m=1 --cluster ceph 2019-12-09T09:13:36.652 INFO:teuthology.orchestra.run.smithi035:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2019-12-09T09:13:39.224 INFO:teuthology.orchestra.run.smithi035.stderr:pool 'default.rgw.buckets.data' created 2019-12-09T09:13:39.239 INFO:teuthology.orchestra.run.smithi035:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2019-12-09T09:13:40.601 INFO:teuthology.orchestra.run.smithi035.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2019-12-09T09:13:40.620 INFO:teuthology.orchestra.run.smithi035:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2019-12-09T09:13:42.368 INFO:teuthology.orchestra.run.smithi035.stderr:pool 'default.rgw.buckets.index' created 2019-12-09T09:13:42.383 INFO:teuthology.orchestra.run.smithi035:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2019-12-09T09:13:43.435 INFO:teuthology.orchestra.run.smithi035.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2019-12-09T09:13:43.454 DEBUG:tasks.rgw:Pools created 2019-12-09T09:13:43.454 INFO:tasks.rgw:Configuring compression type = random 2019-12-09T09:13:43.454 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-12-09T09:13:43.454 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-12-09T09:13:43.455 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-12-09T09:13:43.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:43.611+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:46.411 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:46.391+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:46.411 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:46.391+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:46.506 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:46.486+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:46.506 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:46.486+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:46.513 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:46.494+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:46.513 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:46.494+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:46.518 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:46.499+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:47.676 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.656+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:47.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.664+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:47.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.664+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:47.696 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.676+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:47.696 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.676+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:47.705 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.685+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:47.705 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.685+0000 7f98af2ebec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:13:47.706 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.685+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:47.715 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.695+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:47.715 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.695+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:47.726 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.706+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:47.726 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.706+0000 7f98af2ebec0 10 Creating default zonegroup 2019-12-09T09:13:47.726 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.706+0000 7f98af2ebec0 20 get_system_obj_state: rctx=0x7ffe0251f680 obj=:.pools.avail state=0x55af49067140 s->prefetch_data=0 2019-12-09T09:13:47.729 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.710+0000 7f98af2ebec0 10 couldn't find old data placement pools config, setting up new ones for the zone 2019-12-09T09:13:47.947 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.927+0000 7f98af2ebec0 10 WARNING: store->list_zones() returned r=-2 2019-12-09T09:13:47.947 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.927+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:47.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:47.937+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:48.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:48.946+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:48.978 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:48.958+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:48.978 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:48.959+0000 7f98af2ebec0 10 could not read realm id: (2) No such file or directory 2019-12-09T09:13:48.979 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:48.959+0000 7f98af2ebec0 10 WARNING: failed to set zone as default, r=-22 2019-12-09T09:13:48.979 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:48.959+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:49.019 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:48.999+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:49.246 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.224+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:49.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.287+0000 7f98af2ebec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T09:13:49.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.287+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:49.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.305+0000 7f98af2ebec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:13:49.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.305+0000 7f98af2ebec0 20 zonegroup default 2019-12-09T09:13:49.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.305+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:49.336 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.317+0000 7f98af2ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:13:49.336 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.317+0000 7f98af2ebec0 10 Cannot find current period zone using local zone 2019-12-09T09:13:49.337 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.317+0000 7f98af2ebec0 10 Using default name default 2019-12-09T09:13:49.337 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.317+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:49.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.360+0000 7f9893fff700 0 -- 172.21.15.35:0/4017592157 >> [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] conn(0x55af4905cb10 msgr2=0x55af4905efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:13:49.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.360+0000 7f98af2ebec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:13:49.380 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.360+0000 7f98af2ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:13:49.402 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.381+0000 7f98af2ebec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T09:13:49.402 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.381+0000 7f98af2ebec0 20 zone default found 2019-12-09T09:13:49.402 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.381+0000 7f98af2ebec0 20 started sync module instance, tier type = 2019-12-09T09:13:49.402 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:49.381+0000 7f98af2ebec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:13:53.698 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:53.678+0000 7f989c515700 0 -- 172.21.15.35:0/4017592157 >> [v2:172.21.15.22:6822/9812,v1:172.21.15.22:6824/9812] conn(0x7f988400a550 msgr2=0x7f988400c970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:13:54.906 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:54.886+0000 7f98af2ebec0 20 add_watcher() i=0 2019-12-09T09:13:54.906 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:54.886+0000 7f98af2ebec0 20 add_watcher() i=1 2019-12-09T09:13:54.906 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:54.886+0000 7f98af2ebec0 20 add_watcher() i=2 2019-12-09T09:13:54.906 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:54.886+0000 7f98af2ebec0 20 add_watcher() i=3 2019-12-09T09:13:54.907 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:54.886+0000 7f98af2ebec0 20 add_watcher() i=4 2019-12-09T09:13:54.907 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:54.886+0000 7f98af2ebec0 20 add_watcher() i=5 2019-12-09T09:13:54.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:54.940+0000 7f98af2ebec0 20 add_watcher() i=6 2019-12-09T09:13:54.990 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:54.971+0000 7f98af2ebec0 20 add_watcher() i=7 2019-12-09T09:13:54.990 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:54.971+0000 7f98af2ebec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:13:54.991 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:54.971+0000 7f986bfef700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:13:59.676 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:13:59.655+0000 7f98af2ebec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2019-12-09T09:14:03.890 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:03.869+0000 7f98af2ebec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.log 2019-12-09T09:14:07.758 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:07.738+0000 7f97d27fc700 20 reqs_thread_entry: start 2019-12-09T09:14:07.758 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:07.738+0000 7f97c3fff700 20 reqs_thread_entry: start 2019-12-09T09:14:07.758 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:07.738+0000 7f98af2ebec0 20 init_complete bucket index max shards: 0 2019-12-09T09:14:07.759 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:07.739+0000 7f97c27fc700 20 reqs_thread_entry: start 2019-12-09T09:14:08.237 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:08.216+0000 7f98af2ebec0 20 remove_watcher() i=0 2019-12-09T09:14:08.237 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:08.216+0000 7f98af2ebec0 2 removed watcher, disabling cache 2019-12-09T09:14:08.332 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:08.312+0000 7f98af2ebec0 20 remove_watcher() i=1 2019-12-09T09:14:08.393 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:08.373+0000 7f98af2ebec0 20 remove_watcher() i=2 2019-12-09T09:14:08.478 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:08.458+0000 7f98af2ebec0 20 remove_watcher() i=3 2019-12-09T09:14:08.570 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:08.550+0000 7f98af2ebec0 20 remove_watcher() i=4 2019-12-09T09:14:08.645 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:08.625+0000 7f98af2ebec0 20 remove_watcher() i=5 2019-12-09T09:14:08.757 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:08.737+0000 7f98af2ebec0 20 remove_watcher() i=6 2019-12-09T09:14:08.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:08.814+0000 7f98af2ebec0 20 remove_watcher() i=7 2019-12-09T09:14:08.847 INFO:teuthology.orchestra.run.smithi035.stdout:[] 2019-12-09T09:14:08.848 DEBUG:tasks.util.rgw: json result: [] 2019-12-09T09:14:08.848 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-12-09T09:14:08.848 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-12-09T09:14:08.848 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2019-12-09T09:14:09.137 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.116+0000 7f458affd700 20 reqs_thread_entry: start 2019-12-09T09:14:09.137 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.116+0000 7f465582bec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.151 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.131+0000 7f465582bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:09.151 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.131+0000 7f465582bec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.179 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.158+0000 7f465582bec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T09:14:09.179 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.158+0000 7f465582bec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.189 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.169+0000 7f465582bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:09.189 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.169+0000 7f465582bec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.197 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.177+0000 7f465582bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:09.198 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.177+0000 7f465582bec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.218 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.197+0000 7f465582bec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:14:09.349 INFO:teuthology.orchestra.run.smithi035.stdout:{"id":"d0d8af73-443a-4cb3-a612-7fc7fb9ae85c","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T09:14:09.350 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'd0d8af73-443a-4cb3-a612-7fc7fb9ae85c', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T09:14:09.350 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-12-09T09:14:09.350 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-12-09T09:14:09.351 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-12-09T09:14:09.511 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.491+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.535 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.515+0000 7fd666430ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:09.535 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.515+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.548 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.528+0000 7fd666430ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:09.548 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.528+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.554 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.534+0000 7fd666430ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:09.554 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.534+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.561 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.541+0000 7fd666430ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:09.850 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.830+0000 7fd666430ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:14:09.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.830+0000 7fd666430ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:14:09.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.830+0000 7fd666430ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:14:09.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.830+0000 7fd666430ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:14:09.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.830+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.867 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.847+0000 7fd666430ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:09.867 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.847+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.889 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.869+0000 7fd666430ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:14:09.890 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.869+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.911 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.891+0000 7fd666430ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:09.912 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.891+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.930 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.910+0000 7fd666430ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:14:09.930 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.910+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.938 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.918+0000 7fd666430ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:09.938 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.918+0000 7fd666430ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:14:09.939 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.918+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.949 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.929+0000 7fd666430ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:09.949 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.929+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.946+0000 7fd666430ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.946+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:09.988 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.968+0000 7fd666430ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:14:09.988 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.968+0000 7fd666430ec0 20 zonegroup default 2019-12-09T09:14:09.988 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.968+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:10.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.980+0000 7fd666430ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:10.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.980+0000 7fd666430ec0 10 Cannot find current period zone using local zone 2019-12-09T09:14:10.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.980+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:10.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.990+0000 7fd666430ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:10.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:09.990+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:10.023 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.003+0000 7fd666430ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:10.023 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.003+0000 7fd666430ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:10.045 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.024+0000 7fd666430ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:14:10.045 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.024+0000 7fd666430ec0 20 zone default found 2019-12-09T09:14:10.045 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.024+0000 7fd666430ec0 20 started sync module instance, tier type = 2019-12-09T09:14:10.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.024+0000 7fd666430ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:14:10.521 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.501+0000 7fd666430ec0 20 add_watcher() i=0 2019-12-09T09:14:10.521 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.501+0000 7fd666430ec0 20 add_watcher() i=1 2019-12-09T09:14:10.521 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.501+0000 7fd666430ec0 20 add_watcher() i=2 2019-12-09T09:14:10.521 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.501+0000 7fd666430ec0 20 add_watcher() i=3 2019-12-09T09:14:10.522 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.501+0000 7fd666430ec0 20 add_watcher() i=4 2019-12-09T09:14:10.522 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.501+0000 7fd666430ec0 20 add_watcher() i=5 2019-12-09T09:14:10.522 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.501+0000 7fd666430ec0 20 add_watcher() i=6 2019-12-09T09:14:10.552 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.532+0000 7fd666430ec0 20 add_watcher() i=7 2019-12-09T09:14:10.552 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.532+0000 7fd666430ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:14:10.552 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.532+0000 7fd623fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:14:10.558 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.534+0000 7fd65365a700 0 -- 172.21.15.35:0/3703134993 >> v2:172.21.15.35:3300/0 conn(0x561f8bd21550 msgr2=0x561f8bd239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T09:14:10.593 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:10.573+0000 7fd653e5b700 0 -- 172.21.15.35:0/989274436 >> [v2:172.21.15.22:6800/8484,v1:172.21.15.22:6801/8484] conn(0x7fd594022480 msgr2=0x7fd594024990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:14:13.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.279+0000 7fd5927fc700 20 reqs_thread_entry: start 2019-12-09T09:14:13.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.280+0000 7fd583fff700 20 reqs_thread_entry: start 2019-12-09T09:14:13.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.280+0000 7fd666430ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:14:13.301 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.281+0000 7fd5827fc700 20 reqs_thread_entry: start 2019-12-09T09:14:13.570 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.549+0000 7fd666430ec0 20 remove_watcher() i=0 2019-12-09T09:14:13.570 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.549+0000 7fd666430ec0 2 removed watcher, disabling cache 2019-12-09T09:14:13.643 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.622+0000 7fd666430ec0 20 remove_watcher() i=1 2019-12-09T09:14:13.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.673+0000 7fd666430ec0 20 remove_watcher() i=2 2019-12-09T09:14:13.746 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.726+0000 7fd666430ec0 20 remove_watcher() i=3 2019-12-09T09:14:13.820 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.800+0000 7fd666430ec0 20 remove_watcher() i=4 2019-12-09T09:14:13.874 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.853+0000 7fd666430ec0 20 remove_watcher() i=5 2019-12-09T09:14:13.936 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.916+0000 7fd666430ec0 20 remove_watcher() i=6 2019-12-09T09:14:14.004 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:13.984+0000 7fd666430ec0 20 remove_watcher() i=7 2019-12-09T09:14:14.017 INFO:teuthology.orchestra.run.smithi035.stdout:[] 2019-12-09T09:14:14.017 DEBUG:tasks.util.rgw: json result: [] 2019-12-09T09:14:14.017 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2019-12-09T09:14:14.017 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-12-09T09:14:14.017 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-12-09T09:14:14.018 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2019-12-09T09:14:14.302 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.282+0000 7f61beffd700 20 reqs_thread_entry: start 2019-12-09T09:14:14.302 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.282+0000 7f6289affec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:14.315 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.294+0000 7f6289affec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:14.315 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.294+0000 7f6289affec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:14.327 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.306+0000 7f6289affec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:14.327 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.306+0000 7f6289affec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:14.355 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.331+0000 7f6289affec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:14:14.355 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.331+0000 7f6289affec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:14.368 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.348+0000 7f6289affec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:14:14.445 INFO:teuthology.orchestra.run.smithi035.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2019-12-09T09:14:14.445 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-12-09T09:14:14.445 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-12-09T09:14:14.445 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-12-09T09:14:14.446 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2019-12-09T09:14:14.643 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.623+0000 7f7e55a51ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:14.643 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.623+0000 7f7d8affd700 20 reqs_thread_entry: start 2019-12-09T09:14:14.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.635+0000 7f7e55a51ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:14.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.635+0000 7f7e55a51ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:14.683 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.663+0000 7f7e55a51ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:14:14.683 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.663+0000 7f7e55a51ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:14.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.673+0000 7f7e55a51ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:14.694 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.673+0000 7f7e55a51ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:14.710 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.689+0000 7f7e55a51ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:14.710 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.689+0000 7f7e55a51ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:14.735 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:14.708+0000 7f7e55a51ec0 20 rados_obj.operate() r=0 bl.length=372 2019-12-09T09:14:14.800 INFO:teuthology.orchestra.run.smithi035.stdout:{"id":"d0d8af73-443a-4cb3-a612-7fc7fb9ae85c","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T09:14:14.800 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'd0d8af73-443a-4cb3-a612-7fc7fb9ae85c', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T09:14:14.800 INFO:tasks.rgw:Configuring storage class type = FROZEN 2019-12-09T09:14:14.801 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-12-09T09:14:14.801 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-12-09T09:14:14.801 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2019-12-09T09:14:15.035 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.014+0000 7f9593fff700 20 reqs_thread_entry: start 2019-12-09T09:14:15.035 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.015+0000 7f965ffedec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:15.054 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.029+0000 7f965ffedec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:15.054 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.029+0000 7f965ffedec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:15.064 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.041+0000 7f965ffedec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:15.064 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.041+0000 7f965ffedec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:15.087 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.067+0000 7f965ffedec0 20 rados_obj.operate() r=0 bl.length=372 2019-12-09T09:14:15.088 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.067+0000 7f965ffedec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:15.108 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.087+0000 7f965ffedec0 20 rados_obj.operate() r=0 bl.length=888 2019-12-09T09:14:15.178 INFO:teuthology.orchestra.run.smithi035.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2019-12-09T09:14:15.179 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'FROZEN', u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-12-09T09:14:15.179 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-12-09T09:14:15.179 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-12-09T09:14:15.180 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2019-12-09T09:14:15.397 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.376+0000 7fc5937fe700 20 reqs_thread_entry: start 2019-12-09T09:14:15.397 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.376+0000 7fc65e654ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:15.415 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.394+0000 7fc65e654ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:15.415 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.394+0000 7fc65e654ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:15.445 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.425+0000 7fc65e654ec0 20 rados_obj.operate() r=0 bl.length=888 2019-12-09T09:14:15.446 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.425+0000 7fc65e654ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:15.462 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.442+0000 7fc65e654ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:14:15.463 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.442+0000 7fc65e654ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:15.483 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.458+0000 7fc65e654ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:14:15.483 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.458+0000 7fc65e654ec0 20 rados->read ofs=0 len=0 2019-12-09T09:14:15.495 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:14:15.475+0000 7fc65e654ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:14:15.555 INFO:teuthology.orchestra.run.smithi035.stdout:{"id":"d0d8af73-443a-4cb3-a612-7fc7fb9ae85c","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T09:14:15.556 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'FROZEN': {u'data_pool': u'default.rgw.buckets.data.frozen'}, u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'd0d8af73-443a-4cb3-a612-7fc7fb9ae85c', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T09:14:15.556 INFO:tasks.rgw:Starting rgw... 2019-12-09T09:14:15.557 INFO:tasks.rgw:rgw client.0 config is {'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2019-12-09T09:14:15.557 INFO:tasks.rgw:Using civetweb as radosgw frontend 2019-12-09T09:14:15.557 DEBUG:teuthology.misc:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--tool=memcheck', '--max-threads=1024'] 2019-12-09T09:14:15.558 INFO:tasks.rgw.client.0:Restarting daemon 2019-12-09T09:14:15.558 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2019-12-09T09:14:15.561 INFO:tasks.rgw.client.0:Started 2019-12-09T09:14:15.561 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi035.front.sepia.ceph.com:80/ 2019-12-09T09:14:15.562 INFO:teuthology.orchestra.run.smithi035:> curl http://smithi035.front.sepia.ceph.com:80/ 2019-12-09T09:14:15.630 INFO:teuthology.orchestra.run.smithi035.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:14:15.630 INFO:teuthology.orchestra.run.smithi035.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:14:15.633 INFO:teuthology.orchestra.run.smithi035.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi035.front.sepia.ceph.com:80; Connection refused 2019-12-09T09:14:15.634 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:14:16.635 INFO:teuthology.orchestra.run.smithi035:> curl http://smithi035.front.sepia.ceph.com:80/ 2019-12-09T09:14:16.707 INFO:teuthology.orchestra.run.smithi035.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:14:16.708 INFO:teuthology.orchestra.run.smithi035.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:14:16.711 INFO:teuthology.orchestra.run.smithi035.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi035.front.sepia.ceph.com:80; Connection refused 2019-12-09T09:14:16.712 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:14:18.712 INFO:teuthology.orchestra.run.smithi035:> curl http://smithi035.front.sepia.ceph.com:80/ 2019-12-09T09:14:18.800 INFO:teuthology.orchestra.run.smithi035.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:14:18.800 INFO:teuthology.orchestra.run.smithi035.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:14:18.805 INFO:teuthology.orchestra.run.smithi035.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi035.front.sepia.ceph.com:80; Connection refused 2019-12-09T09:14:18.806 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:14:22.807 INFO:teuthology.orchestra.run.smithi035:> curl http://smithi035.front.sepia.ceph.com:80/ 2019-12-09T09:14:22.913 INFO:teuthology.orchestra.run.smithi035.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:14:22.913 INFO:teuthology.orchestra.run.smithi035.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:14:22.917 INFO:teuthology.orchestra.run.smithi035.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi035.front.sepia.ceph.com:80; Connection refused 2019-12-09T09:14:22.917 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:14:30.918 INFO:teuthology.orchestra.run.smithi035:> curl http://smithi035.front.sepia.ceph.com:80/ 2019-12-09T09:14:30.981 INFO:teuthology.orchestra.run.smithi035.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:14:30.982 INFO:teuthology.orchestra.run.smithi035.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:14:31.595 INFO:teuthology.orchestra.run.smithi035.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 349 0 --:--:-- --:--:-- --:--:-- 348 2019-12-09T09:14:31.596 INFO:teuthology.orchestra.run.smithi035.stdout:anonymous 2019-12-09T09:14:31.596 INFO:teuthology.run_tasks:Running task workunit... 2019-12-09T09:14:31.611 INFO:tasks.workunit:Pulling workunits from ref 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:14:31.611 INFO:tasks.workunit:Making a separate scratch dir for every client... 2019-12-09T09:14:31.611 INFO:teuthology.orchestra.run.smithi035:> stat -- /home/ubuntu/cephtest/mnt.0 2019-12-09T09:14:31.659 INFO:teuthology.orchestra.run.smithi035.stderr:stat: cannot stat ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2019-12-09T09:14:31.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T09:14:31.660 INFO:teuthology.orchestra.run.smithi035:> mkdir -- /home/ubuntu/cephtest/mnt.0 2019-12-09T09:14:31.708 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2019-12-09T09:14:31.708 INFO:teuthology.orchestra.run.smithi035:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2019-12-09T09:14:31.754 INFO:tasks.workunit:timeout=3h 2019-12-09T09:14:31.754 INFO:tasks.workunit:cleanup=True 2019-12-09T09:14:31.754 INFO:teuthology.orchestra.run.smithi035:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:14:32.025 INFO:tasks.workunit.client.0.smithi035.stdout:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2019-12-09T09:15:37.973 INFO:tasks.workunit.client.0.smithi035.stderr:Note: checking out '1351c819e1a9518cde4c8b12d645782009028360'. 2019-12-09T09:15:37.973 INFO:tasks.workunit.client.0.smithi035.stderr: 2019-12-09T09:15:37.974 INFO:tasks.workunit.client.0.smithi035.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2019-12-09T09:15:37.974 INFO:tasks.workunit.client.0.smithi035.stderr:changes and commit them, and you can discard any commits you make in this 2019-12-09T09:15:37.974 INFO:tasks.workunit.client.0.smithi035.stderr:state without impacting any branches by performing another checkout. 2019-12-09T09:15:37.974 INFO:tasks.workunit.client.0.smithi035.stderr: 2019-12-09T09:15:37.974 INFO:tasks.workunit.client.0.smithi035.stderr:If you want to create a new branch to retain commits you create, you may 2019-12-09T09:15:37.974 INFO:tasks.workunit.client.0.smithi035.stderr:do so (now or later) by using -b with the checkout command again. Example: 2019-12-09T09:15:37.974 INFO:tasks.workunit.client.0.smithi035.stderr: 2019-12-09T09:15:37.975 INFO:tasks.workunit.client.0.smithi035.stderr: git checkout -b new_branch_name 2019-12-09T09:15:37.975 INFO:tasks.workunit.client.0.smithi035.stderr: 2019-12-09T09:15:37.975 INFO:tasks.workunit.client.0.smithi035.stderr:HEAD is now at 1351c81... rgw: Fix dynamic resharding not working for empty zonegroup in period 2019-12-09T09:15:37.976 INFO:teuthology.orchestra.run.smithi035:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2019-12-09T09:15:38.048 INFO:tasks.workunit.client.0.smithi035.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2019-12-09T09:15:38.050 INFO:tasks.workunit.client.0.smithi035.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-12-09T09:15:38.051 INFO:tasks.workunit.client.0.smithi035.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2019-12-09T09:15:38.514 INFO:tasks.workunit.client.0.smithi035.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2019-12-09T09:15:38.609 INFO:tasks.workunit.client.0.smithi035.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2019-12-09T09:15:38.642 INFO:tasks.workunit.client.0.smithi035.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-12-09T09:15:38.661 INFO:tasks.workunit.client.0.smithi035.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-12-09T09:15:38.661 INFO:tasks.workunit.client.0.smithi035.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2019-12-09T09:15:38.699 INFO:tasks.workunit.client.0.smithi035.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-12-09T09:15:38.761 DEBUG:teuthology.orchestra.remote:smithi035:/home/ubuntu/cephtest/workunits.list.client.0 is 4KB 2019-12-09T09:15:38.816 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2019-12-09T09:15:38.816 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2019-12-09T09:15:38.816 INFO:teuthology.orchestra.run.smithi035:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2019-12-09T09:15:38.874 INFO:tasks.workunit.client.0.smithi035.stdout:Running main() from gmock_main.cc 2019-12-09T09:15:38.875 INFO:tasks.workunit.client.0.smithi035.stdout:[==========] Running 11 tests from 1 test suite. 2019-12-09T09:15:38.875 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] Global test environment set-up. 2019-12-09T09:15:38.875 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] 11 tests from ClsLock 2019-12-09T09:15:38.875 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] ClsLock.TestMultiLocking 2019-12-09T09:15:41.603 INFO:tasks.workunit.client.0.smithi035.stdout:lock: mylock 2019-12-09T09:15:41.603 INFO:tasks.workunit.client.0.smithi035.stdout: lock_type: exclusive 2019-12-09T09:15:41.603 INFO:tasks.workunit.client.0.smithi035.stdout: tag: 2019-12-09T09:15:41.604 INFO:tasks.workunit.client.0.smithi035.stdout: lockers: 2019-12-09T09:15:41.604 INFO:tasks.workunit.client.0.smithi035.stdout: client.4364 expiration=2019-12-09T09:17:41.356151+0000 addr=v1:172.21.15.35:0/624802652 cookie= 2019-12-09T09:15:41.817 INFO:tasks.workunit.client.0.smithi035.stdout:lock: mylock 2019-12-09T09:15:41.817 INFO:tasks.workunit.client.0.smithi035.stdout: lock_type: shared 2019-12-09T09:15:41.817 INFO:tasks.workunit.client.0.smithi035.stdout: tag: 2019-12-09T09:15:41.817 INFO:tasks.workunit.client.0.smithi035.stdout: lockers: 2019-12-09T09:15:41.818 INFO:tasks.workunit.client.0.smithi035.stdout: client.4364 expiration=2019-12-09T09:17:41.722097+0000 addr=v1:172.21.15.35:0/624802652 cookie= 2019-12-09T09:15:41.818 INFO:tasks.workunit.client.0.smithi035.stdout: client.4367 expiration=0.000000 addr=v1:172.21.15.35:0/2263944340 cookie= 2019-12-09T09:15:41.867 INFO:tasks.workunit.client.0.smithi035.stdout:lock: mylock 2019-12-09T09:15:41.867 INFO:tasks.workunit.client.0.smithi035.stdout: lock_type: shared 2019-12-09T09:15:41.867 INFO:tasks.workunit.client.0.smithi035.stdout: tag: 2019-12-09T09:15:41.867 INFO:tasks.workunit.client.0.smithi035.stdout: lockers: 2019-12-09T09:15:41.867 INFO:tasks.workunit.client.0.smithi035.stdout: client.4367 expiration=0.000000 addr=v1:172.21.15.35:0/2263944340 cookie= 2019-12-09T09:15:43.151 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] ClsLock.TestMultiLocking (4277 ms) 2019-12-09T09:15:43.151 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] ClsLock.TestMeta 2019-12-09T09:15:46.020 INFO:tasks.workunit.client.0.smithi035.stdout:lock: mylock 2019-12-09T09:15:46.020 INFO:tasks.workunit.client.0.smithi035.stdout: lock_type: shared 2019-12-09T09:15:46.021 INFO:tasks.workunit.client.0.smithi035.stdout: tag: 2019-12-09T09:15:46.021 INFO:tasks.workunit.client.0.smithi035.stdout: lockers: 2019-12-09T09:15:46.021 INFO:tasks.workunit.client.0.smithi035.stdout: client.4379 expiration=0.000000 addr=v1:172.21.15.35:0/2003601685 cookie= 2019-12-09T09:15:46.094 INFO:tasks.workunit.client.0.smithi035.stdout:lock: mylock 2019-12-09T09:15:46.094 INFO:tasks.workunit.client.0.smithi035.stdout: lock_type: exclusive 2019-12-09T09:15:46.094 INFO:tasks.workunit.client.0.smithi035.stdout: tag: new_tag 2019-12-09T09:15:46.095 INFO:tasks.workunit.client.0.smithi035.stdout: lockers: 2019-12-09T09:15:46.095 INFO:tasks.workunit.client.0.smithi035.stdout: client.4373 expiration=0.000000 addr=v1:172.21.15.35:0/2512415165 cookie= 2019-12-09T09:15:46.560 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] ClsLock.TestMeta (3409 ms) 2019-12-09T09:15:46.561 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] ClsLock.TestCookie 2019-12-09T09:15:49.574 INFO:tasks.workunit.client.0.smithi035.stdout:lock: mylock 2019-12-09T09:15:49.574 INFO:tasks.workunit.client.0.smithi035.stdout: lock_type: exclusive 2019-12-09T09:15:49.574 INFO:tasks.workunit.client.0.smithi035.stdout: tag: 2019-12-09T09:15:49.574 INFO:tasks.workunit.client.0.smithi035.stdout: lockers: 2019-12-09T09:15:49.649 INFO:tasks.workunit.client.0.smithi035.stdout:lock: mylock 2019-12-09T09:15:49.649 INFO:tasks.workunit.client.0.smithi035.stdout: lock_type: shared 2019-12-09T09:15:49.649 INFO:tasks.workunit.client.0.smithi035.stdout: tag: 2019-12-09T09:15:49.650 INFO:tasks.workunit.client.0.smithi035.stdout: lockers: 2019-12-09T09:15:49.650 INFO:tasks.workunit.client.0.smithi035.stdout: client.4561 expiration=0.000000 addr=v1:172.21.15.35:0/35032387 cookie= 2019-12-09T09:15:49.650 INFO:tasks.workunit.client.0.smithi035.stdout: client.4561 expiration=0.000000 addr=v1:172.21.15.35:0/35032387 cookie=new cookie 2019-12-09T09:15:50.182 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] ClsLock.TestCookie (3622 ms) 2019-12-09T09:15:50.182 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] ClsLock.TestMultipleLocks 2019-12-09T09:15:53.587 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] ClsLock.TestMultipleLocks (3405 ms) 2019-12-09T09:15:53.587 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] ClsLock.TestLockDuration 2019-12-09T09:16:02.168 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] ClsLock.TestLockDuration (8580 ms) 2019-12-09T09:16:02.168 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] ClsLock.TestAssertLocked 2019-12-09T09:16:05.900 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] ClsLock.TestAssertLocked (3733 ms) 2019-12-09T09:16:05.901 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] ClsLock.TestSetCookie 2019-12-09T09:16:09.368 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] ClsLock.TestSetCookie (3465 ms) 2019-12-09T09:16:09.368 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] ClsLock.TestRenew 2019-12-09T09:16:32.178 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] ClsLock.TestRenew (22811 ms) 2019-12-09T09:16:32.178 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2019-12-09T09:16:39.601 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (7423 ms) 2019-12-09T09:16:39.602 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2019-12-09T09:16:47.565 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (7963 ms) 2019-12-09T09:16:47.565 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2019-12-09T09:16:55.997 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (8431 ms) 2019-12-09T09:16:55.997 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] 11 tests from ClsLock (77119 ms total) 2019-12-09T09:16:55.997 INFO:tasks.workunit.client.0.smithi035.stdout: 2019-12-09T09:16:55.997 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] Global test environment tear-down 2019-12-09T09:16:55.997 INFO:tasks.workunit.client.0.smithi035.stdout:[==========] 11 tests from 1 test suite ran. (77119 ms total) 2019-12-09T09:16:55.998 INFO:tasks.workunit.client.0.smithi035.stdout:[ PASSED ] 11 tests. 2019-12-09T09:16:55.999 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:16:55.999 INFO:teuthology.orchestra.run.smithi035:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:16:56.109 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2019-12-09T09:16:56.109 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2019-12-09T09:16:56.109 INFO:teuthology.orchestra.run.smithi035:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2019-12-09T09:16:56.166 INFO:tasks.workunit.client.0.smithi035.stdout:Running main() from gmock_main.cc 2019-12-09T09:16:56.167 INFO:tasks.workunit.client.0.smithi035.stdout:[==========] Running 4 tests from 1 test suite. 2019-12-09T09:16:56.167 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] Global test environment set-up. 2019-12-09T09:16:56.167 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] 4 tests from cls_log 2019-12-09T09:16:56.167 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_log.test_log_add_same_time 2019-12-09T09:16:59.499 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_log.test_log_add_same_time (3333 ms) 2019-12-09T09:16:59.500 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_log.test_log_add_different_time 2019-12-09T09:17:03.217 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_log.test_log_add_different_time (3717 ms) 2019-12-09T09:17:03.217 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_log.trim_by_time 2019-12-09T09:17:08.821 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_log.trim_by_time (5604 ms) 2019-12-09T09:17:08.822 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_log.trim_by_marker 2019-12-09T09:17:12.136 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_log.trim_by_marker (3315 ms) 2019-12-09T09:17:12.136 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] 4 tests from cls_log (15969 ms total) 2019-12-09T09:17:12.136 INFO:tasks.workunit.client.0.smithi035.stdout: 2019-12-09T09:17:12.136 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] Global test environment tear-down 2019-12-09T09:17:12.136 INFO:tasks.workunit.client.0.smithi035.stdout:[==========] 4 tests from 1 test suite ran. (15969 ms total) 2019-12-09T09:17:12.136 INFO:tasks.workunit.client.0.smithi035.stdout:[ PASSED ] 4 tests. 2019-12-09T09:17:12.138 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:17:12.138 INFO:teuthology.orchestra.run.smithi035:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:17:12.208 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2019-12-09T09:17:12.208 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2019-12-09T09:17:12.208 INFO:teuthology.orchestra.run.smithi035:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2019-12-09T09:17:12.264 INFO:tasks.workunit.client.0.smithi035.stdout:Running main() from gmock_main.cc 2019-12-09T09:17:12.264 INFO:tasks.workunit.client.0.smithi035.stdout:[==========] Running 5 tests from 1 test suite. 2019-12-09T09:17:12.264 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] Global test environment set-up. 2019-12-09T09:17:12.265 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] 5 tests from cls_rgw 2019-12-09T09:17:12.265 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.test_implicit 2019-12-09T09:17:15.414 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.test_implicit (3150 ms) 2019-12-09T09:17:15.414 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.test_implicit_idempotent 2019-12-09T09:17:19.015 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.test_implicit_idempotent (3601 ms) 2019-12-09T09:17:19.015 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.test_put_snap 2019-12-09T09:17:28.089 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T09:17:28.020+0000 3aa81700 -1 osd.2 pg_epoch: 104 pg[25.6( v 103'4 (0'0,103'4] local-lis/les=100/101 n=2 ec=100/100 lis/c=100/100 les/c/f=101/101/0 sis=100) [2,1] r=0 lpr=100 crt=102'3 lcod 102'3 mlcod 102'3 active+clean+snaptrim trimq=[1~1]] removing snap head 2019-12-09T09:17:29.730 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.test_put_snap (10714 ms) 2019-12-09T09:17:29.730 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.test_explicit 2019-12-09T09:17:33.250 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.test_explicit (3519 ms) 2019-12-09T09:17:33.250 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.set 2019-12-09T09:17:38.094 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.set (4844 ms) 2019-12-09T09:17:38.094 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] 5 tests from cls_rgw (25828 ms total) 2019-12-09T09:17:38.094 INFO:tasks.workunit.client.0.smithi035.stdout: 2019-12-09T09:17:38.094 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] Global test environment tear-down 2019-12-09T09:17:38.095 INFO:tasks.workunit.client.0.smithi035.stdout:[==========] 5 tests from 1 test suite ran. (25828 ms total) 2019-12-09T09:17:38.095 INFO:tasks.workunit.client.0.smithi035.stdout:[ PASSED ] 5 tests. 2019-12-09T09:17:38.096 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:17:38.096 INFO:teuthology.orchestra.run.smithi035:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:17:38.216 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2019-12-09T09:17:38.216 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2019-12-09T09:17:38.216 INFO:teuthology.orchestra.run.smithi035:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2019-12-09T09:17:38.279 INFO:tasks.workunit.client.0.smithi035.stdout:Running main() from gmock_main.cc 2019-12-09T09:17:38.279 INFO:tasks.workunit.client.0.smithi035.stdout:[==========] Running 13 tests from 1 test suite. 2019-12-09T09:17:38.279 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] Global test environment set-up. 2019-12-09T09:17:38.279 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] 13 tests from cls_rgw 2019-12-09T09:17:40.740 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.index_basic 2019-12-09T09:17:42.273 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.index_basic (1532 ms) 2019-12-09T09:17:42.273 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2019-12-09T09:17:44.111 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (1838 ms) 2019-12-09T09:17:44.111 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.index_remove_object 2019-12-09T09:17:46.221 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.index_remove_object (2110 ms) 2019-12-09T09:17:46.221 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.index_suggest 2019-12-09T09:18:12.165 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.index_suggest (25942 ms) 2019-12-09T09:18:12.165 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.index_list 2019-12-09T09:18:13.714 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.index_list (1550 ms) 2019-12-09T09:18:13.714 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.bi_list 2019-12-09T09:18:16.564 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.bi_list (2849 ms) 2019-12-09T09:18:16.564 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.gc_set 2019-12-09T09:18:17.284 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.gc_set (720 ms) 2019-12-09T09:18:17.284 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.gc_list 2019-12-09T09:18:17.849 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.gc_list (565 ms) 2019-12-09T09:18:17.849 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.gc_defer 2019-12-09T09:18:25.607 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.gc_defer (7757 ms) 2019-12-09T09:18:25.607 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.usage_basic 2019-12-09T09:18:34.694 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.usage_basic (9087 ms) 2019-12-09T09:18:34.694 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2019-12-09T09:18:34.782 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.usage_clear_no_obj (88 ms) 2019-12-09T09:18:34.782 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.usage_clear 2019-12-09T09:18:42.735 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.usage_clear (7953 ms) 2019-12-09T09:18:42.735 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] cls_rgw.bi_log_trim 2019-12-09T09:18:44.652 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] cls_rgw.bi_log_trim (1916 ms) 2019-12-09T09:18:45.534 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] 13 tests from cls_rgw (63907 ms total) 2019-12-09T09:18:45.535 INFO:tasks.workunit.client.0.smithi035.stdout: 2019-12-09T09:18:45.535 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] Global test environment tear-down 2019-12-09T09:18:45.535 INFO:tasks.workunit.client.0.smithi035.stdout:[==========] 13 tests from 1 test suite ran. (67250 ms total) 2019-12-09T09:18:45.535 INFO:tasks.workunit.client.0.smithi035.stdout:[ PASSED ] 13 tests. 2019-12-09T09:18:45.537 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:18:45.537 INFO:teuthology.orchestra.run.smithi035:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:18:45.664 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2019-12-09T09:18:45.664 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2019-12-09T09:18:45.664 INFO:teuthology.orchestra.run.smithi035:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2019-12-09T09:18:45.752 INFO:tasks.workunit.client.0.smithi035.stdout:[==========] Running 6 tests from 1 test suite. 2019-12-09T09:18:45.752 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] Global test environment set-up. 2019-12-09T09:18:47.582 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] 6 tests from rgw_gc_log 2019-12-09T09:18:47.582 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2019-12-09T09:18:48.354 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] rgw_gc_log.init_existing_queue (769 ms) 2019-12-09T09:18:48.354 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2019-12-09T09:18:48.450 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] rgw_gc_log.init_existing_omap (100 ms) 2019-12-09T09:18:48.451 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2019-12-09T09:18:48.567 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (113 ms) 2019-12-09T09:18:48.567 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2019-12-09T09:18:48.635 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (71 ms) 2019-12-09T09:18:48.636 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2019-12-09T09:18:48.740 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] rgw_gc_log.defer1_after_init (106 ms) 2019-12-09T09:18:48.741 INFO:tasks.workunit.client.0.smithi035.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2019-12-09T09:18:48.825 INFO:tasks.workunit.client.0.smithi035.stdout:[ OK ] rgw_gc_log.defer2_before_init (63 ms) 2019-12-09T09:18:48.825 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] 6 tests from rgw_gc_log (1222 ms total) 2019-12-09T09:18:48.826 INFO:tasks.workunit.client.0.smithi035.stdout: 2019-12-09T09:18:48.826 INFO:tasks.workunit.client.0.smithi035.stdout:[----------] Global test environment tear-down 2019-12-09T09:18:49.955 INFO:tasks.workunit.client.0.smithi035.stdout:[==========] 6 tests from 1 test suite ran. (4208 ms total) 2019-12-09T09:18:49.955 INFO:tasks.workunit.client.0.smithi035.stdout:[ PASSED ] 6 tests. 2019-12-09T09:18:49.958 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:18:49.958 INFO:teuthology.orchestra.run.smithi035:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:18:50.031 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh'] on client.0... 2019-12-09T09:18:50.031 INFO:teuthology.orchestra.run.smithi035:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2019-12-09T09:18:50.245 DEBUG:teuthology.parallel:result is None 2019-12-09T09:18:50.246 INFO:teuthology.orchestra.run.smithi035:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T09:18:50.310 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T09:18:50.310 INFO:teuthology.orchestra.run.smithi035:> rmdir -- /home/ubuntu/cephtest/mnt.0 2019-12-09T09:18:50.388 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T09:18:50.388 INFO:teuthology.run_tasks:Running task ragweed... 2019-12-09T09:18:50.419 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}} 2019-12-09T09:18:50.420 INFO:tasks.ragweed:Downloading ragweed... 2019-12-09T09:18:50.420 INFO:tasks.ragweed:Using branch 'wip-ofriedma-dynamic-disable' for ragweed 2019-12-09T09:18:50.420 INFO:teuthology.orchestra.run.smithi035:> git clone -b wip-ofriedma-dynamic-disable git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-12-09T09:18:50.498 DEBUG:teuthology.orchestra.run:got remote process result: 128 2019-12-09T09:18:50.498 INFO:teuthology.orchestra.run.smithi035.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-12-09T09:18:50.498 INFO:teuthology.orchestra.run.smithi035.stderr:fatal: Remote branch wip-ofriedma-dynamic-disable not found in upstream origin 2019-12-09T09:18:50.499 INFO:teuthology.orchestra.run.smithi035:> git clone -b ceph-master git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-12-09T09:18:50.605 INFO:teuthology.orchestra.run.smithi035.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-12-09T09:18:50.719 INFO:tasks.ragweed:Creating rgw users... 2019-12-09T09:18:50.719 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2019-12-09T09:18:50.719 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key ZFXYDUXWOSUPFELXOEXO --secret akC97mXQx6cumH/UnBTjSe1h5d2mP+z3geqqFdPuJo/xq0FA9JS9dA== --email sysuser.client.0+test@test.test --system true 2019-12-09T09:18:50.840 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:50.806+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:50.861 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:50.827+0000 7fbfabb11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:50.861 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:50.827+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:50.870 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:50.836+0000 7fbfabb11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:50.870 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:50.836+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:50.874 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:50.841+0000 7fbfabb11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:50.874 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:50.841+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:50.883 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:50.849+0000 7fbfabb11ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:18:51.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.035+0000 7fbfabb11ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:18:51.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.035+0000 7fbfabb11ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:18:51.070 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.035+0000 7fbfabb11ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:18:51.070 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.035+0000 7fbfabb11ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:18:51.070 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.035+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.043+0000 7fbfabb11ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:18:51.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.043+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.098 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.065+0000 7fbfabb11ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:18:51.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.065+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.107 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.074+0000 7fbfabb11ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:18:51.107 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.074+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.125 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.091+0000 7fbfabb11ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:18:51.125 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.091+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.135 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.101+0000 7fbfabb11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:51.135 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.101+0000 7fbfabb11ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:18:51.135 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.101+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.142 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.108+0000 7fbfabb11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:51.142 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.108+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.151 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.117+0000 7fbfabb11ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:18:51.151 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.117+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.166 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.133+0000 7fbfabb11ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:18:51.167 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.133+0000 7fbfabb11ec0 20 zonegroup default 2019-12-09T09:18:51.167 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.133+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.172 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.138+0000 7fbfabb11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:51.172 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.138+0000 7fbfabb11ec0 10 Cannot find current period zone using local zone 2019-12-09T09:18:51.172 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.138+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.179 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.146+0000 7fbfabb11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:51.180 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.146+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.186 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.153+0000 7fbfabb11ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:18:51.186 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.153+0000 7fbfabb11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:51.206 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.172+0000 7fbfabb11ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:18:51.206 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.172+0000 7fbfabb11ec0 20 zone default found 2019-12-09T09:18:51.206 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.172+0000 7fbfabb11ec0 20 started sync module instance, tier type = 2019-12-09T09:18:51.206 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.172+0000 7fbfabb11ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:18:51.578 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.543+0000 7fbfabb11ec0 20 add_watcher() i=0 2019-12-09T09:18:51.578 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.543+0000 7fbfabb11ec0 20 add_watcher() i=1 2019-12-09T09:18:51.578 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.543+0000 7fbfabb11ec0 20 add_watcher() i=2 2019-12-09T09:18:51.578 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.543+0000 7fbfabb11ec0 20 add_watcher() i=3 2019-12-09T09:18:51.578 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.543+0000 7fbfabb11ec0 20 add_watcher() i=4 2019-12-09T09:18:51.578 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.543+0000 7fbfabb11ec0 20 add_watcher() i=5 2019-12-09T09:18:51.602 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.568+0000 7fbfabb11ec0 20 add_watcher() i=6 2019-12-09T09:18:51.615 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.581+0000 7fbfabb11ec0 20 add_watcher() i=7 2019-12-09T09:18:51.615 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.581+0000 7fbfabb11ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:18:51.615 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.581+0000 7fbf68fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:18:51.617 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.584+0000 7fbf93fff700 0 -- 172.21.15.35:0/535497068 >> v1:172.21.15.35:6789/0 conn(0x55d48a66af10 legacy=0x55d48a66d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T09:18:51.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:51.625+0000 7fbf98d3b700 0 -- 172.21.15.35:0/47590623 >> [v2:172.21.15.35:6808/9789,v1:172.21.15.35:6809/9789] conn(0x55d48a67d430 msgr2=0x55d48a67f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T09:18:53.538 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.501+0000 7fbedaffd700 20 reqs_thread_entry: start 2019-12-09T09:18:53.538 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.502+0000 7fbed8ff9700 20 reqs_thread_entry: start 2019-12-09T09:18:53.538 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.502+0000 7fbfabb11ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:18:53.539 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.502+0000 7fbeceffd700 20 reqs_thread_entry: start 2019-12-09T09:18:53.539 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.502+0000 7fbfabb11ec0 20 get_system_obj_state: rctx=0x7ffdcc3ad468 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x55d48a6e9f00 s->prefetch_data=0 2019-12-09T09:18:53.539 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.502+0000 7fbfabb11ec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-12-09T09:18:53.558 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.523+0000 7fbfabb11ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2019-12-09T09:18:53.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.523+0000 7fbfabb11ec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:18:53.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.524+0000 7fbfabb11ec0 20 get_system_obj_state: rctx=0x7ffdcc3ad478 obj=default.rgw.meta:users.email:sysuser.client.0+test@test.test state=0x55d48a6ebc90 s->prefetch_data=0 2019-12-09T09:18:53.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.524+0000 7fbfabb11ec0 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2019-12-09T09:18:53.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.539+0000 7fbfabb11ec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2019-12-09T09:18:53.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.539+0000 7fbfabb11ec0 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T09:18:53.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.539+0000 7fbfabb11ec0 20 get_system_obj_state: rctx=0x7ffdcc3ad478 obj=default.rgw.meta:users.keys:ZFXYDUXWOSUPFELXOEXO state=0x55d48a6ebc90 s->prefetch_data=0 2019-12-09T09:18:53.575 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.539+0000 7fbfabb11ec0 10 cache get: name=default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO : miss 2019-12-09T09:18:53.591 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.556+0000 7fbfabb11ec0 10 cache put: name=default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO info.flags=0x0 2019-12-09T09:18:53.591 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.556+0000 7fbfabb11ec0 10 adding default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO to cache LRU end 2019-12-09T09:18:53.592 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.556+0000 7fbfabb11ec0 20 get_system_obj_state: rctx=0x7ffdcc3ad088 obj=default.rgw.meta:users.keys:ZFXYDUXWOSUPFELXOEXO state=0x55d48a6ebc90 s->prefetch_data=0 2019-12-09T09:18:53.592 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.556+0000 7fbfabb11ec0 10 cache get: name=default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO : type miss (requested=0x6, cached=0x0) 2019-12-09T09:18:53.600 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.565+0000 7fbfabb11ec0 10 cache put: name=default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO info.flags=0x0 2019-12-09T09:18:53.600 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.565+0000 7fbfabb11ec0 10 moving default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO to cache LRU end 2019-12-09T09:18:53.600 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.565+0000 7fbfabb11ec0 20 get_system_obj_state: rctx=0x7ffdcc3ad208 obj=default.rgw.meta:users.keys:ZFXYDUXWOSUPFELXOEXO state=0x55d48a6ebc90 s->prefetch_data=0 2019-12-09T09:18:53.601 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.565+0000 7fbfabb11ec0 10 cache get: name=default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO : type miss (requested=0x6, cached=0x0) 2019-12-09T09:18:53.612 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.577+0000 7fbfabb11ec0 10 cache put: name=default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO info.flags=0x0 2019-12-09T09:18:53.612 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.577+0000 7fbfabb11ec0 10 moving default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO to cache LRU end 2019-12-09T09:18:53.687 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.652+0000 7fbfabb11ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-12-09T09:18:53.687 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.652+0000 7fbfabb11ec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:18:53.687 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.652+0000 7fbfabb11ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=637 2019-12-09T09:18:53.733 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.696+0000 7fbf81ffb700 10 RGWWatcher::handle_notify() notify_id 536870912000 cookie 94371343301952 notifier 4675 bl.length()=637 2019-12-09T09:18:53.733 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.696+0000 7fbf81ffb700 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-12-09T09:18:53.734 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.696+0000 7fbf81ffb700 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:18:53.995 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.960+0000 7fbfabb11ec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-12-09T09:18:53.995 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.960+0000 7fbfabb11ec0 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T09:18:53.995 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:53.960+0000 7fbfabb11ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=202 2019-12-09T09:18:54.052 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.017+0000 7fbf81ffb700 10 RGWWatcher::handle_notify() notify_id 536870912000 cookie 94371343318576 notifier 4675 bl.length()=202 2019-12-09T09:18:54.052 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.017+0000 7fbf81ffb700 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-12-09T09:18:54.053 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.017+0000 7fbf81ffb700 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T09:18:54.241 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.206+0000 7fbfabb11ec0 10 cache put: name=default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO info.flags=0x7 2019-12-09T09:18:54.241 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.206+0000 7fbfabb11ec0 10 moving default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO to cache LRU end 2019-12-09T09:18:54.241 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.206+0000 7fbfabb11ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=190 2019-12-09T09:18:54.250 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.215+0000 7fbf81ffb700 10 RGWWatcher::handle_notify() notify_id 536870912001 cookie 94371343318576 notifier 4675 bl.length()=190 2019-12-09T09:18:54.250 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.215+0000 7fbf81ffb700 10 cache put: name=default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO info.flags=0x7 2019-12-09T09:18:54.250 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.215+0000 7fbf81ffb700 10 moving default.rgw.meta+users.keys+ZFXYDUXWOSUPFELXOEXO to cache LRU end 2019-12-09T09:18:54.806 INFO:teuthology.orchestra.run.smithi035.stdout:{ 2019-12-09T09:18:54.806 INFO:teuthology.orchestra.run.smithi035.stdout: "user_id": "sysuser.client.0", 2019-12-09T09:18:54.806 INFO:teuthology.orchestra.run.smithi035.stdout: "display_name": "Mr. sysuser.client.0", 2019-12-09T09:18:54.807 INFO:teuthology.orchestra.run.smithi035.stdout: "email": "sysuser.client.0+test@test.test", 2019-12-09T09:18:54.807 INFO:teuthology.orchestra.run.smithi035.stdout: "suspended": 0, 2019-12-09T09:18:54.807 INFO:teuthology.orchestra.run.smithi035.stdout: "max_buckets": 1000, 2019-12-09T09:18:54.807 INFO:teuthology.orchestra.run.smithi035.stdout: "subusers": [], 2019-12-09T09:18:54.807 INFO:teuthology.orchestra.run.smithi035.stdout: "keys": [ 2019-12-09T09:18:54.807 INFO:teuthology.orchestra.run.smithi035.stdout: { 2019-12-09T09:18:54.807 INFO:teuthology.orchestra.run.smithi035.stdout: "user": "sysuser.client.0", 2019-12-09T09:18:54.808 INFO:teuthology.orchestra.run.smithi035.stdout: "access_key": "ZFXYDUXWOSUPFELXOEXO", 2019-12-09T09:18:54.808 INFO:teuthology.orchestra.run.smithi035.stdout: "secret_key": "akC97mXQx6cumH/UnBTjSe1h5d2mP+z3geqqFdPuJo/xq0FA9JS9dA==" 2019-12-09T09:18:54.808 INFO:teuthology.orchestra.run.smithi035.stdout: } 2019-12-09T09:18:54.808 INFO:teuthology.orchestra.run.smithi035.stdout: ], 2019-12-09T09:18:54.808 INFO:teuthology.orchestra.run.smithi035.stdout: "swift_keys": [], 2019-12-09T09:18:54.808 INFO:teuthology.orchestra.run.smithi035.stdout: "caps": [], 2019-12-09T09:18:54.808 INFO:teuthology.orchestra.run.smithi035.stdout: "op_mask": "read, write, delete", 2019-12-09T09:18:54.809 INFO:teuthology.orchestra.run.smithi035.stdout: "system": "true", 2019-12-09T09:18:54.809 INFO:teuthology.orchestra.run.smithi035.stdout: "default_placement": "", 2019-12-09T09:18:54.809 INFO:teuthology.orchestra.run.smithi035.stdout: "default_storage_class": "", 2019-12-09T09:18:54.809 INFO:teuthology.orchestra.run.smithi035.stdout: "placement_tags": [], 2019-12-09T09:18:54.809 INFO:teuthology.orchestra.run.smithi035.stdout: "bucket_quota": { 2019-12-09T09:18:54.809 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T09:18:54.809 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T09:18:54.809 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T09:18:54.810 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T09:18:54.810 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T09:18:54.810 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T09:18:54.810 INFO:teuthology.orchestra.run.smithi035.stdout: "user_quota": { 2019-12-09T09:18:54.810 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T09:18:54.810 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T09:18:54.810 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T09:18:54.811 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T09:18:54.811 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T09:18:54.811 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T09:18:54.811 INFO:teuthology.orchestra.run.smithi035.stdout: "temp_url_keys": [], 2019-12-09T09:18:54.811 INFO:teuthology.orchestra.run.smithi035.stdout: "type": "rgw", 2019-12-09T09:18:54.811 INFO:teuthology.orchestra.run.smithi035.stdout: "mfa_ids": [] 2019-12-09T09:18:54.811 INFO:teuthology.orchestra.run.smithi035.stdout:} 2019-12-09T09:18:54.811 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:18:54.881 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.847+0000 7fbfabb11ec0 20 remove_watcher() i=0 2019-12-09T09:18:54.882 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.847+0000 7fbfabb11ec0 2 removed watcher, disabling cache 2019-12-09T09:18:54.947 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.912+0000 7fbfabb11ec0 20 remove_watcher() i=1 2019-12-09T09:18:55.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:54.967+0000 7fbfabb11ec0 20 remove_watcher() i=2 2019-12-09T09:18:55.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.034+0000 7fbfabb11ec0 20 remove_watcher() i=3 2019-12-09T09:18:55.137 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.103+0000 7fbfabb11ec0 20 remove_watcher() i=4 2019-12-09T09:18:55.197 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.162+0000 7fbfabb11ec0 20 remove_watcher() i=5 2019-12-09T09:18:55.256 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.222+0000 7fbfabb11ec0 20 remove_watcher() i=6 2019-12-09T09:18:55.308 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.274+0000 7fbfabb11ec0 20 remove_watcher() i=7 2019-12-09T09:18:55.318 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2019-12-09T09:18:55.319 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key YAQERBREYAHHWLIDJILD --secret E+Qv2fP40Lft0bhNcbceWMYhirZUza0SR+oQexxDt4e+m1vY3AfXiQ== --email ragweed.client.0+test@test.test --system false 2019-12-09T09:18:55.447 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.413+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.471 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.435+0000 7fbee6c11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:55.471 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.435+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.481 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.445+0000 7fbee6c11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:55.481 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.445+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.487 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.453+0000 7fbee6c11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:55.487 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.453+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.495 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.461+0000 7fbee6c11ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:18:55.754 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.720+0000 7fbee6c11ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:18:55.755 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.720+0000 7fbee6c11ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:18:55.755 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.720+0000 7fbee6c11ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:18:55.755 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.720+0000 7fbee6c11ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:18:55.755 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.720+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.766 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.732+0000 7fbee6c11ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:18:55.766 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.732+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.792 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.752+0000 7fbee6c11ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:18:55.792 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.752+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.768+0000 7fbee6c11ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:18:55.803 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.769+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.821 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.786+0000 7fbee6c11ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:18:55.821 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.786+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.831 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.797+0000 7fbee6c11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:55.831 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.797+0000 7fbee6c11ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:18:55.831 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.797+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.838 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.804+0000 7fbee6c11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:55.838 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.804+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.854 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.820+0000 7fbee6c11ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:18:55.854 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.820+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.854+0000 7fbee6c11ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:18:55.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.854+0000 7fbee6c11ec0 20 zonegroup default 2019-12-09T09:18:55.889 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.854+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.896 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.861+0000 7fbee6c11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:55.896 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.861+0000 7fbee6c11ec0 10 Cannot find current period zone using local zone 2019-12-09T09:18:55.896 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.861+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.869+0000 7fbee6c11ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:18:55.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.869+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.914 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.880+0000 7fbee6c11ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:18:55.914 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.880+0000 7fbee6c11ec0 20 rados->read ofs=0 len=0 2019-12-09T09:18:55.937 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.898+0000 7fbee6c11ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:18:55.937 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.898+0000 7fbee6c11ec0 20 zone default found 2019-12-09T09:18:55.938 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.898+0000 7fbee6c11ec0 20 started sync module instance, tier type = 2019-12-09T09:18:55.938 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:55.898+0000 7fbee6c11ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:18:56.313 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.279+0000 7fbee6c11ec0 20 add_watcher() i=0 2019-12-09T09:18:56.314 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.279+0000 7fbee6c11ec0 20 add_watcher() i=1 2019-12-09T09:18:56.314 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.279+0000 7fbee6c11ec0 20 add_watcher() i=2 2019-12-09T09:18:56.314 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.279+0000 7fbee6c11ec0 20 add_watcher() i=3 2019-12-09T09:18:56.314 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.279+0000 7fbee6c11ec0 20 add_watcher() i=4 2019-12-09T09:18:56.314 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.279+0000 7fbee6c11ec0 20 add_watcher() i=5 2019-12-09T09:18:56.315 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.279+0000 7fbee6c11ec0 20 add_watcher() i=6 2019-12-09T09:18:56.346 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.312+0000 7fbee6c11ec0 20 add_watcher() i=7 2019-12-09T09:18:56.346 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.312+0000 7fbee6c11ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:18:56.347 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.312+0000 7fbe27fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:18:56.367 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.333+0000 7fbed363a700 0 -- 172.21.15.35:0/2283093290 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x56499d313a00 msgr2=0x56499d315f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:18:56.410 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:56.375+0000 7fbed463c700 0 -- 172.21.15.35:0/2283093290 >> [v2:172.21.15.22:6800/8484,v1:172.21.15.22:6801/8484] conn(0x7fbe1402a9b0 msgr2=0x7fbe1402ce40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:18:58.055 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.016+0000 7fbe127fc700 20 reqs_thread_entry: start 2019-12-09T09:18:58.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.016+0000 7fbe0bfff700 20 reqs_thread_entry: start 2019-12-09T09:18:58.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.017+0000 7fbee6c11ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:18:58.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.018+0000 7fbe0a7fc700 20 reqs_thread_entry: start 2019-12-09T09:18:58.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.018+0000 7fbee6c11ec0 20 get_system_obj_state: rctx=0x7ffe72507628 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x56499d388a00 s->prefetch_data=0 2019-12-09T09:18:58.058 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.018+0000 7fbee6c11ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-12-09T09:18:58.059 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.024+0000 7fbee6c11ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2019-12-09T09:18:58.059 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.024+0000 7fbee6c11ec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:18:58.059 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.024+0000 7fbee6c11ec0 20 get_system_obj_state: rctx=0x7ffe72507638 obj=default.rgw.meta:users.email:ragweed.client.0+test@test.test state=0x56499d38a790 s->prefetch_data=0 2019-12-09T09:18:58.060 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.024+0000 7fbee6c11ec0 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2019-12-09T09:18:58.067 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.030+0000 7fbee6c11ec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2019-12-09T09:18:58.068 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.030+0000 7fbee6c11ec0 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T09:18:58.068 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.030+0000 7fbee6c11ec0 20 get_system_obj_state: rctx=0x7ffe72507638 obj=default.rgw.meta:users.keys:YAQERBREYAHHWLIDJILD state=0x56499d38a790 s->prefetch_data=0 2019-12-09T09:18:58.068 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.030+0000 7fbee6c11ec0 10 cache get: name=default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD : miss 2019-12-09T09:18:58.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.043+0000 7fbee6c11ec0 10 cache put: name=default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD info.flags=0x0 2019-12-09T09:18:58.078 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.043+0000 7fbee6c11ec0 10 adding default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD to cache LRU end 2019-12-09T09:18:58.078 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.043+0000 7fbee6c11ec0 20 get_system_obj_state: rctx=0x7ffe72507248 obj=default.rgw.meta:users.keys:YAQERBREYAHHWLIDJILD state=0x56499d38a790 s->prefetch_data=0 2019-12-09T09:18:58.078 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.043+0000 7fbee6c11ec0 10 cache get: name=default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD : type miss (requested=0x6, cached=0x0) 2019-12-09T09:18:58.089 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.054+0000 7fbee6c11ec0 10 cache put: name=default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD info.flags=0x0 2019-12-09T09:18:58.090 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.054+0000 7fbee6c11ec0 10 moving default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD to cache LRU end 2019-12-09T09:18:58.090 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.054+0000 7fbee6c11ec0 20 get_system_obj_state: rctx=0x7ffe725073c8 obj=default.rgw.meta:users.keys:YAQERBREYAHHWLIDJILD state=0x56499d38a790 s->prefetch_data=0 2019-12-09T09:18:58.090 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.054+0000 7fbee6c11ec0 10 cache get: name=default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD : type miss (requested=0x6, cached=0x0) 2019-12-09T09:18:58.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.063+0000 7fbee6c11ec0 10 cache put: name=default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD info.flags=0x0 2019-12-09T09:18:58.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.063+0000 7fbee6c11ec0 10 moving default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD to cache LRU end 2019-12-09T09:18:58.140 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.106+0000 7fbee6c11ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-12-09T09:18:58.140 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.106+0000 7fbee6c11ec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:18:58.141 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.106+0000 7fbee6c11ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=637 2019-12-09T09:18:58.151 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.116+0000 7fbec97fa700 10 RGWWatcher::handle_notify() notify_id 536870912002 cookie 94874169721584 notifier 4681 bl.length()=637 2019-12-09T09:18:58.151 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.116+0000 7fbec97fa700 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-12-09T09:18:58.151 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.116+0000 7fbec97fa700 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:18:58.527 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.493+0000 7fbee6c11ec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-12-09T09:18:58.527 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.493+0000 7fbee6c11ec0 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T09:18:58.527 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.493+0000 7fbee6c11ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=202 2019-12-09T09:18:58.540 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.506+0000 7fbec97fa700 10 RGWWatcher::handle_notify() notify_id 536870912001 cookie 94874169703888 notifier 4681 bl.length()=202 2019-12-09T09:18:58.540 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.506+0000 7fbec97fa700 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-12-09T09:18:58.540 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.506+0000 7fbec97fa700 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T09:18:58.810 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.775+0000 7fbee6c11ec0 10 cache put: name=default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD info.flags=0x7 2019-12-09T09:18:58.810 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.775+0000 7fbee6c11ec0 10 moving default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD to cache LRU end 2019-12-09T09:18:58.810 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.775+0000 7fbee6c11ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=190 2019-12-09T09:18:58.864 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.830+0000 7fbec97fa700 10 RGWWatcher::handle_notify() notify_id 536870912000 cookie 94874169706832 notifier 4681 bl.length()=190 2019-12-09T09:18:58.864 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.830+0000 7fbec97fa700 10 cache put: name=default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD info.flags=0x7 2019-12-09T09:18:58.864 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:58.830+0000 7fbec97fa700 10 moving default.rgw.meta+users.keys+YAQERBREYAHHWLIDJILD to cache LRU end 2019-12-09T09:18:59.052 INFO:teuthology.orchestra.run.smithi035.stdout:{ 2019-12-09T09:18:59.052 INFO:teuthology.orchestra.run.smithi035.stdout: "user_id": "ragweed.client.0", 2019-12-09T09:18:59.053 INFO:teuthology.orchestra.run.smithi035.stdout: "display_name": "Mr. ragweed.client.0", 2019-12-09T09:18:59.053 INFO:teuthology.orchestra.run.smithi035.stdout: "email": "ragweed.client.0+test@test.test", 2019-12-09T09:18:59.053 INFO:teuthology.orchestra.run.smithi035.stdout: "suspended": 0, 2019-12-09T09:18:59.053 INFO:teuthology.orchestra.run.smithi035.stdout: "max_buckets": 1000, 2019-12-09T09:18:59.054 INFO:teuthology.orchestra.run.smithi035.stdout: "subusers": [], 2019-12-09T09:18:59.055 INFO:teuthology.orchestra.run.smithi035.stdout: "keys": [ 2019-12-09T09:18:59.056 INFO:teuthology.orchestra.run.smithi035.stdout: { 2019-12-09T09:18:59.056 INFO:teuthology.orchestra.run.smithi035.stdout: "user": "ragweed.client.0", 2019-12-09T09:18:59.056 INFO:teuthology.orchestra.run.smithi035.stdout: "access_key": "YAQERBREYAHHWLIDJILD", 2019-12-09T09:18:59.056 INFO:teuthology.orchestra.run.smithi035.stdout: "secret_key": "E+Qv2fP40Lft0bhNcbceWMYhirZUza0SR+oQexxDt4e+m1vY3AfXiQ==" 2019-12-09T09:18:59.057 INFO:teuthology.orchestra.run.smithi035.stdout: } 2019-12-09T09:18:59.057 INFO:teuthology.orchestra.run.smithi035.stdout: ], 2019-12-09T09:18:59.057 INFO:teuthology.orchestra.run.smithi035.stdout: "swift_keys": [], 2019-12-09T09:18:59.058 INFO:teuthology.orchestra.run.smithi035.stdout: "caps": [], 2019-12-09T09:18:59.058 INFO:teuthology.orchestra.run.smithi035.stdout: "op_mask": "read, write, delete", 2019-12-09T09:18:59.058 INFO:teuthology.orchestra.run.smithi035.stdout: "system": "true", 2019-12-09T09:18:59.058 INFO:teuthology.orchestra.run.smithi035.stdout: "default_placement": "", 2019-12-09T09:18:59.059 INFO:teuthology.orchestra.run.smithi035.stdout: "default_storage_class": "", 2019-12-09T09:18:59.059 INFO:teuthology.orchestra.run.smithi035.stdout: "placement_tags": [], 2019-12-09T09:18:59.059 INFO:teuthology.orchestra.run.smithi035.stdout: "bucket_quota": { 2019-12-09T09:18:59.059 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T09:18:59.060 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T09:18:59.060 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T09:18:59.060 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T09:18:59.060 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T09:18:59.060 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T09:18:59.061 INFO:teuthology.orchestra.run.smithi035.stdout: "user_quota": { 2019-12-09T09:18:59.061 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T09:18:59.061 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T09:18:59.061 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T09:18:59.062 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T09:18:59.062 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T09:18:59.062 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T09:18:59.062 INFO:teuthology.orchestra.run.smithi035.stdout: "temp_url_keys": [], 2019-12-09T09:18:59.063 INFO:teuthology.orchestra.run.smithi035.stdout: "type": "rgw", 2019-12-09T09:18:59.063 INFO:teuthology.orchestra.run.smithi035.stdout: "mfa_ids": [] 2019-12-09T09:18:59.063 INFO:teuthology.orchestra.run.smithi035.stdout:} 2019-12-09T09:18:59.063 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:18:59.119 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:59.085+0000 7fbee6c11ec0 20 remove_watcher() i=0 2019-12-09T09:18:59.119 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:59.085+0000 7fbee6c11ec0 2 removed watcher, disabling cache 2019-12-09T09:18:59.180 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:59.146+0000 7fbee6c11ec0 20 remove_watcher() i=1 2019-12-09T09:18:59.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:59.189+0000 7fbee6c11ec0 20 remove_watcher() i=2 2019-12-09T09:18:59.305 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:59.265+0000 7fbee6c11ec0 20 remove_watcher() i=3 2019-12-09T09:18:59.375 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:59.340+0000 7fbee6c11ec0 20 remove_watcher() i=4 2019-12-09T09:18:59.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:59.387+0000 7fbee6c11ec0 20 remove_watcher() i=5 2019-12-09T09:18:59.468 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:59.433+0000 7fbee6c11ec0 20 remove_watcher() i=6 2019-12-09T09:18:59.505 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:18:59.471+0000 7fbee6c11ec0 20 remove_watcher() i=7 2019-12-09T09:18:59.515 INFO:tasks.ragweed:Configuring ragweed... 2019-12-09T09:18:59.515 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest/ragweed && ./bootstrap 2019-12-09T09:19:06.051 INFO:teuthology.orchestra.run.smithi035.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-12-09T09:19:06.176 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:19:06.221 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:19:06.222 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T09:19:06.222 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:19:06.223 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:19:06.223 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T09:19:06.877 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:19:07.226 INFO:teuthology.orchestra.run.smithi035.stdout:Package python2-pip-8.1.2-10.el7.noarch already installed and latest version 2019-12-09T09:19:08.353 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T09:19:08.353 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:19:08.354 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package libevent-devel.x86_64 0:2.0.21-4.el7 will be installed 2019-12-09T09:19:08.356 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 will be installed 2019-12-09T09:19:08.356 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: xz-devel for package: libxml2-devel-2.9.1-6.el7_2.3.x86_64 2019-12-09T09:19:08.390 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package libxslt-devel.x86_64 0:1.1.28-5.el7 will be installed 2019-12-09T09:19:08.392 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libgcrypt-devel for package: libxslt-devel-1.1.28-5.el7.x86_64 2019-12-09T09:19:08.393 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed 2019-12-09T09:19:08.394 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:19:08.394 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package libgcrypt-devel.x86_64 0:1.5.3-14.el7 will be installed 2019-12-09T09:19:08.396 INFO:teuthology.orchestra.run.smithi035.stdout:--> Processing Dependency: libgpg-error-devel for package: libgcrypt-devel-1.5.3-14.el7.x86_64 2019-12-09T09:19:08.399 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package xz-devel.x86_64 0:5.2.2-1.el7 will be installed 2019-12-09T09:19:08.400 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T09:19:08.400 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package libgpg-error-devel.x86_64 0:1.12-3.el7 will be installed 2019-12-09T09:19:08.802 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T09:19:09.000 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:09.000 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T09:19:09.001 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:09.001 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:19:09.001 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2019-12-09T09:19:09.001 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:19:09.001 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T09:19:09.001 INFO:teuthology.orchestra.run.smithi035.stdout: libevent-devel x86_64 2.0.21-4.el7 base 85 k 2019-12-09T09:19:09.002 INFO:teuthology.orchestra.run.smithi035.stdout: libxml2-devel x86_64 2.9.1-6.el7_2.3 base 1.0 M 2019-12-09T09:19:09.002 INFO:teuthology.orchestra.run.smithi035.stdout: libxslt-devel x86_64 1.1.28-5.el7 base 309 k 2019-12-09T09:19:09.002 INFO:teuthology.orchestra.run.smithi035.stdout: zlib-devel x86_64 1.2.7-18.el7 base 50 k 2019-12-09T09:19:09.002 INFO:teuthology.orchestra.run.smithi035.stdout:Installing for dependencies: 2019-12-09T09:19:09.002 INFO:teuthology.orchestra.run.smithi035.stdout: libgcrypt-devel x86_64 1.5.3-14.el7 base 129 k 2019-12-09T09:19:09.002 INFO:teuthology.orchestra.run.smithi035.stdout: libgpg-error-devel x86_64 1.12-3.el7 base 16 k 2019-12-09T09:19:09.003 INFO:teuthology.orchestra.run.smithi035.stdout: xz-devel x86_64 5.2.2-1.el7 base 46 k 2019-12-09T09:19:09.003 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:09.003 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T09:19:09.003 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T09:19:09.003 INFO:teuthology.orchestra.run.smithi035.stdout:Install 4 Packages (+3 Dependent packages) 2019-12-09T09:19:09.003 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:09.004 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 1.7 M 2019-12-09T09:19:09.004 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 12 M 2019-12-09T09:19:09.004 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T09:19:09.265 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:19:09.265 INFO:teuthology.orchestra.run.smithi035.stdout:Total 6.3 MB/s | 1.7 MB 00:00 2019-12-09T09:19:09.275 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T09:19:09.291 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T09:19:09.357 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T09:19:09.357 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T09:19:10.200 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libgpg-error-devel-1.12-3.el7.x86_64 1/7 2019-12-09T09:19:10.509 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libgcrypt-devel-1.5.3-14.el7.x86_64 2/7 2019-12-09T09:19:10.824 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-12-09T09:19:11.406 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : xz-devel-5.2.2-1.el7.x86_64 4/7 2019-12-09T09:19:11.878 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libxml2-devel-2.9.1-6.el7_2.3.x86_64 5/7 2019-12-09T09:19:12.326 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libxslt-devel-1.1.28-5.el7.x86_64 6/7 2019-12-09T09:19:12.613 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libevent-devel-2.0.21-4.el7.x86_64 7/7 2019-12-09T09:19:12.747 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : xz-devel-5.2.2-1.el7.x86_64 1/7 2019-12-09T09:19:12.905 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libxml2-devel-2.9.1-6.el7_2.3.x86_64 2/7 2019-12-09T09:19:13.098 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-12-09T09:19:13.232 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libxslt-devel-1.1.28-5.el7.x86_64 4/7 2019-12-09T09:19:13.366 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libgpg-error-devel-1.12-3.el7.x86_64 5/7 2019-12-09T09:19:13.516 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libevent-devel-2.0.21-4.el7.x86_64 6/7 2019-12-09T09:19:13.989 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libgcrypt-devel-1.5.3-14.el7.x86_64 7/7 2019-12-09T09:19:13.989 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:13.989 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T09:19:13.990 INFO:teuthology.orchestra.run.smithi035.stdout: libevent-devel.x86_64 0:2.0.21-4.el7 libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 2019-12-09T09:19:13.990 INFO:teuthology.orchestra.run.smithi035.stdout: libxslt-devel.x86_64 0:1.1.28-5.el7 zlib-devel.x86_64 0:1.2.7-18.el7 2019-12-09T09:19:13.990 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:13.990 INFO:teuthology.orchestra.run.smithi035.stdout:Dependency Installed: 2019-12-09T09:19:13.990 INFO:teuthology.orchestra.run.smithi035.stdout: libgcrypt-devel.x86_64 0:1.5.3-14.el7 libgpg-error-devel.x86_64 0:1.12-3.el7 2019-12-09T09:19:13.990 INFO:teuthology.orchestra.run.smithi035.stdout: xz-devel.x86_64 0:5.2.2-1.el7 2019-12-09T09:19:13.991 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:13.991 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T09:19:14.220 INFO:teuthology.orchestra.run.smithi035.stdout:New python executable in /home/ubuntu/cephtest/ragweed/virtualenv/bin/python 2019-12-09T09:19:15.950 INFO:teuthology.orchestra.run.smithi035.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T09:19:16.514 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting pip 2019-12-09T09:19:16.615 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl (1.4MB) 2019-12-09T09:19:16.900 INFO:teuthology.orchestra.run.smithi035.stdout:Installing collected packages: pip 2019-12-09T09:19:16.900 INFO:teuthology.orchestra.run.smithi035.stdout: Found existing installation: pip 9.0.1 2019-12-09T09:19:16.993 INFO:teuthology.orchestra.run.smithi035.stdout: Uninstalling pip-9.0.1: 2019-12-09T09:19:17.048 INFO:teuthology.orchestra.run.smithi035.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T09:19:17.824 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully installed pip-19.3.1 2019-12-09T09:19:18.871 INFO:teuthology.orchestra.run.smithi035.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:19:18.967 INFO:teuthology.orchestra.run.smithi035.stdout:Requirement already satisfied: six in /usr/lib/python2.7/site-packages (1.9.0) 2019-12-09T09:19:19.856 INFO:teuthology.orchestra.run.smithi035.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:19:20.039 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting nose 2019-12-09T09:19:20.086 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl (154kB) 2019-12-09T09:19:20.219 INFO:teuthology.orchestra.run.smithi035.stdout:Installing collected packages: nose 2019-12-09T09:19:20.295 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully installed nose-1.3.7 2019-12-09T09:19:20.804 INFO:teuthology.orchestra.run.smithi035.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:19:21.408 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting setuptools==32.3.1 2019-12-09T09:19:21.456 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl (479kB) 2019-12-09T09:19:21.616 INFO:teuthology.orchestra.run.smithi035.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-12-09T09:19:21.616 INFO:teuthology.orchestra.run.smithi035.stdout:Installing collected packages: setuptools 2019-12-09T09:19:21.616 INFO:teuthology.orchestra.run.smithi035.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T09:19:21.636 INFO:teuthology.orchestra.run.smithi035.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T09:19:21.641 INFO:teuthology.orchestra.run.smithi035.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T09:19:21.770 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully installed setuptools-32.3.1 2019-12-09T09:19:22.299 INFO:teuthology.orchestra.run.smithi035.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:19:22.609 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting PyYAML 2019-12-09T09:19:22.648 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/8d/c9/e5be955a117a1ac548cdd31e37e8fd7b02ce987f9655f5c7563c656d5dcb/PyYAML-5.2.tar.gz (265kB) 2019-12-09T09:19:23.396 INFO:teuthology.orchestra.run.smithi035.stdout:Requirement already up-to-date: nose>=1.0.0 in ./virtualenv/lib/python2.7/site-packages (from -r requirements.txt (line 2)) (1.3.7) 2019-12-09T09:19:23.532 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting boto!=2.46.0,>=2.6.0 2019-12-09T09:19:23.543 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl (1.4MB) 2019-12-09T09:19:23.861 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting bunch>=1.0.0 2019-12-09T09:19:23.872 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/ef/bf/a4cf1779a4ffb4f610903fa08e15d1f4a8a2f4e3353a02afbe097c5bf4a8/bunch-1.0.1.tar.gz 2019-12-09T09:19:24.581 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting gevent>=1.0 2019-12-09T09:19:24.596 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (5.0MB) 2019-12-09T09:19:25.038 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting httplib2 2019-12-09T09:19:25.050 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/ce/2e/87461bfbb7e561203b759b3f7f639e2144226604372830d00a8279960ae1/httplib2-0.14.0.tar.gz (219kB) 2019-12-09T09:19:26.397 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting lxml 2019-12-09T09:19:26.415 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl (5.7MB) 2019-12-09T09:19:26.919 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T09:19:26.929 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T09:19:27.028 INFO:teuthology.orchestra.run.smithi035.stdout:Building wheels for collected packages: PyYAML, bunch, httplib2 2019-12-09T09:19:27.029 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for PyYAML (setup.py): started 2019-12-09T09:19:27.814 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for PyYAML (setup.py): finished with status 'done' 2019-12-09T09:19:27.815 INFO:teuthology.orchestra.run.smithi035.stdout: Created wheel for PyYAML: filename=PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl size=46299 sha256=0b77740893b02c8aca8eaa5462b6df06b24b1ccdac5fbbf6dbe0673fc122d4d3 2019-12-09T09:19:27.815 INFO:teuthology.orchestra.run.smithi035.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67 2019-12-09T09:19:27.849 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for bunch (setup.py): started 2019-12-09T09:19:28.168 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for bunch (setup.py): finished with status 'done' 2019-12-09T09:19:28.168 INFO:teuthology.orchestra.run.smithi035.stdout: Created wheel for bunch: filename=bunch-1.0.1-cp27-none-any.whl size=8347 sha256=f03fe2a1ff3f4fa334c0acd4e41fada6ba7c6cee3cc2828344f3c4f278689329 2019-12-09T09:19:28.168 INFO:teuthology.orchestra.run.smithi035.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c 2019-12-09T09:19:28.172 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for httplib2 (setup.py): started 2019-12-09T09:19:28.588 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for httplib2 (setup.py): finished with status 'done' 2019-12-09T09:19:28.590 INFO:teuthology.orchestra.run.smithi035.stdout: Created wheel for httplib2: filename=httplib2-0.14.0-cp27-none-any.whl size=96742 sha256=305a629ea99eafcbfd09b5469d0f260f2b7ef38be0823ff5d080bf9e5f063737 2019-12-09T09:19:28.590 INFO:teuthology.orchestra.run.smithi035.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488 2019-12-09T09:19:28.604 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully built PyYAML bunch httplib2 2019-12-09T09:19:28.738 INFO:teuthology.orchestra.run.smithi035.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-12-09T09:19:28.738 INFO:teuthology.orchestra.run.smithi035.stdout:Installing collected packages: PyYAML, boto, bunch, greenlet, gevent, httplib2, lxml 2019-12-09T09:19:28.739 INFO:teuthology.orchestra.run.smithi035.stdout: Found existing installation: PyYAML 3.10 2019-12-09T09:19:28.739 INFO:teuthology.orchestra.run.smithi035.stdout: Not uninstalling pyyaml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-12-09T09:19:28.739 INFO:teuthology.orchestra.run.smithi035.stdout: Can't uninstall 'PyYAML'. No files were found to uninstall. 2019-12-09T09:19:30.067 INFO:teuthology.orchestra.run.smithi035.stdout: Found existing installation: lxml 3.2.1 2019-12-09T09:19:30.068 INFO:teuthology.orchestra.run.smithi035.stdout: Not uninstalling lxml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-12-09T09:19:30.068 INFO:teuthology.orchestra.run.smithi035.stdout: Can't uninstall 'lxml'. No files were found to uninstall. 2019-12-09T09:19:30.139 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully installed PyYAML-5.2 boto-2.49.0 bunch-1.0.1 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 lxml-4.4.2 2019-12-09T09:19:30.569 INFO:teuthology.orchestra.run.smithi035.stdout:running develop 2019-12-09T09:19:30.641 INFO:teuthology.orchestra.run.smithi035.stdout:running egg_info 2019-12-09T09:19:30.641 INFO:teuthology.orchestra.run.smithi035.stdout:creating ragweed.egg-info 2019-12-09T09:19:30.641 INFO:teuthology.orchestra.run.smithi035.stdout:writing requirements to ragweed.egg-info/requires.txt 2019-12-09T09:19:30.641 INFO:teuthology.orchestra.run.smithi035.stdout:writing ragweed.egg-info/PKG-INFO 2019-12-09T09:19:30.641 INFO:teuthology.orchestra.run.smithi035.stdout:writing top-level names to ragweed.egg-info/top_level.txt 2019-12-09T09:19:30.641 INFO:teuthology.orchestra.run.smithi035.stdout:writing dependency_links to ragweed.egg-info/dependency_links.txt 2019-12-09T09:19:30.642 INFO:teuthology.orchestra.run.smithi035.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T09:19:30.642 INFO:teuthology.orchestra.run.smithi035.stdout:reading manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T09:19:30.642 INFO:teuthology.orchestra.run.smithi035.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T09:19:30.644 INFO:teuthology.orchestra.run.smithi035.stdout:running build_ext 2019-12-09T09:19:30.644 INFO:teuthology.orchestra.run.smithi035.stdout:Creating /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/ragweed.egg-link (link to .) 2019-12-09T09:19:30.644 INFO:teuthology.orchestra.run.smithi035.stdout:Adding ragweed 0.0.1 to easy-install.pth file 2019-12-09T09:19:30.645 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:30.645 INFO:teuthology.orchestra.run.smithi035.stdout:Installed /home/ubuntu/cephtest/ragweed 2019-12-09T09:19:30.645 INFO:teuthology.orchestra.run.smithi035.stdout:Processing dependencies for ragweed==0.0.1 2019-12-09T09:19:30.649 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for isodate>=0.4.4 2019-12-09T09:19:30.649 INFO:teuthology.orchestra.run.smithi035.stdout:Reading https://pypi.python.org/simple/isodate/ 2019-12-09T09:19:30.742 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading https://files.pythonhosted.org/packages/b1/80/fb8c13a4cd38eb5021dc3741a9e588e4d1de88d895c1910c6fc8a08b7a70/isodate-0.6.0.tar.gz#sha256=2e364a3d5759479cdb2d37cce6b9376ea504db2ff90252a2e5b7cc89cc9ff2d8 2019-12-09T09:19:30.795 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: isodate 0.6.0 2019-12-09T09:19:30.795 INFO:teuthology.orchestra.run.smithi035.stdout:Processing isodate-0.6.0.tar.gz 2019-12-09T09:19:30.804 INFO:teuthology.orchestra.run.smithi035.stdout:Writing /tmp/easy_install-C067iz/isodate-0.6.0/setup.cfg 2019-12-09T09:19:30.804 INFO:teuthology.orchestra.run.smithi035.stdout:Running isodate-0.6.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-C067iz/isodate-0.6.0/egg-dist-tmp-5G0Pt1 2019-12-09T09:19:31.090 INFO:teuthology.orchestra.run.smithi035.stderr:zip_safe flag not set; analyzing archive contents... 2019-12-09T09:19:31.105 INFO:teuthology.orchestra.run.smithi035.stdout:Moving isodate-0.6.0-py2.7.egg to /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:19:31.107 INFO:teuthology.orchestra.run.smithi035.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-12-09T09:19:31.108 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:31.108 INFO:teuthology.orchestra.run.smithi035.stdout:Installed /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/isodate-0.6.0-py2.7.egg 2019-12-09T09:19:31.131 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for gevent==1.4.0 2019-12-09T09:19:31.132 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: gevent 1.4.0 2019-12-09T09:19:31.132 INFO:teuthology.orchestra.run.smithi035.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-12-09T09:19:31.132 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:31.132 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:19:31.133 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for bunch==1.0.1 2019-12-09T09:19:31.133 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: bunch 1.0.1 2019-12-09T09:19:31.133 INFO:teuthology.orchestra.run.smithi035.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-12-09T09:19:31.134 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:31.134 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:19:31.135 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for PyYAML==5.2 2019-12-09T09:19:31.135 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: PyYAML 5.2 2019-12-09T09:19:31.135 INFO:teuthology.orchestra.run.smithi035.stdout:Adding PyYAML 5.2 to easy-install.pth file 2019-12-09T09:19:31.136 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:31.136 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:19:31.136 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for boto==2.49.0 2019-12-09T09:19:31.136 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: boto 2.49.0 2019-12-09T09:19:31.136 INFO:teuthology.orchestra.run.smithi035.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-12-09T09:19:31.136 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:31.136 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:19:31.137 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for six==1.9.0 2019-12-09T09:19:31.137 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: six 1.9.0 2019-12-09T09:19:31.137 INFO:teuthology.orchestra.run.smithi035.stdout:Adding six 1.9.0 to easy-install.pth file 2019-12-09T09:19:31.138 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:31.138 INFO:teuthology.orchestra.run.smithi035.stdout:Using /usr/lib/python2.7/site-packages 2019-12-09T09:19:31.143 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for greenlet==0.4.15 2019-12-09T09:19:31.143 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: greenlet 0.4.15 2019-12-09T09:19:31.144 INFO:teuthology.orchestra.run.smithi035.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-12-09T09:19:31.144 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:19:31.144 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:19:31.144 INFO:teuthology.orchestra.run.smithi035.stdout:Finished processing dependencies for ragweed==0.0.1 2019-12-09T09:19:31.172 INFO:teuthology.orchestra.run.smithi035:> cat > /home/ubuntu/cephtest/archive/ragweed.client.0.conf 2019-12-09T09:19:31.243 INFO:tasks.ragweed:Configuring boto... 2019-12-09T09:19:31.244 INFO:teuthology.orchestra.run.smithi035:> cat > /home/ubuntu/cephtest/boto.cfg 2019-12-09T09:19:31.348 INFO:teuthology.orchestra.run.smithi035:ragweed tests against rgw> RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw' 2019-12-09T09:20:04.463 INFO:teuthology.orchestra.run.smithi035.stderr:ragweed.tests.tests.r_test_multipart_defer_complete.test ... ok 2019-12-09T09:20:29.788 INFO:teuthology.orchestra.run.smithi035.stderr:ragweed.tests.tests.r_test_multipart_defer_update_complete.test ... ok 2019-12-09T09:20:55.836 INFO:teuthology.orchestra.run.smithi035.stderr:ragweed.tests.tests.r_test_multipart_index_versioning.test ... ok 2019-12-09T09:21:17.386 INFO:teuthology.orchestra.run.smithi035.stderr:ragweed.tests.tests.r_test_multipart_simple.test ... ok 2019-12-09T09:21:36.113 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:09:12.384 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:21:36.171 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:09:12.455 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:21:36.213 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:09:12.485 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:21:36.231 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:09:12.516 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:21:36.237 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:09:12.522 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:21:36.280 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:09:12.564 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:21:36.798 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:09:13.081 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:21:36.862 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:09:13.146 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:21:36.953 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:09:13.225 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:21:47.275 INFO:teuthology.orchestra.run.smithi035.stderr:ragweed.tests.tests.r_test_obj_storage_class.test ... ok 2019-12-09T09:23:02.004 INFO:teuthology.orchestra.run.smithi035.stderr:ragweed.tests.tests.r_test_obj_storage_class_copy.test ... ok 2019-12-09T09:23:36.609 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:11:12.883 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:23:36.625 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:11:12.899 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:23:36.626 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:11:12.902 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:23:36.635 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:11:12.911 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:23:36.636 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:11:12.912 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:23:36.640 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:11:12.917 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:23:36.765 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:11:13.038 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:23:36.805 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:11:13.081 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:23:36.808 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:11:13.084 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:24:05.444 INFO:teuthology.orchestra.run.smithi035.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart.test ... ok 2019-12-09T09:25:57.319 INFO:teuthology.orchestra.run.smithi035.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart_copy.test ... ok 2019-12-09T09:26:31.524 INFO:teuthology.orchestra.run.smithi035.stderr:ragweed.tests.tests.r_test_small_obj_data.test ... ok 2019-12-09T09:26:31.525 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T09:26:31.525 INFO:teuthology.orchestra.run.smithi035.stderr:---------------------------------------------------------------------- 2019-12-09T09:26:31.525 INFO:teuthology.orchestra.run.smithi035.stderr:Ran 9 tests in 419.701s 2019-12-09T09:26:31.525 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T09:26:31.525 INFO:teuthology.orchestra.run.smithi035.stderr:OK 2019-12-09T09:26:31.551 INFO:tasks.ragweed:Cleaning up boto... 2019-12-09T09:26:31.552 INFO:teuthology.orchestra.run.smithi035:> rm /home/ubuntu/cephtest/boto.cfg 2019-12-09T09:26:31.634 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2019-12-09T09:26:31.770 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:31.704+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:31.794 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:31.729+0000 7fe6c708dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:31.795 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:31.729+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:31.806 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:31.740+0000 7fe6c708dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:31.806 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:31.740+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:31.813 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:31.748+0000 7fe6c708dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:31.813 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:31.748+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:31.824 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:31.759+0000 7fe6c708dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:26:32.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.005+0000 7fe6c708dec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:26:32.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.005+0000 7fe6c708dec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:26:32.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.005+0000 7fe6c708dec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:26:32.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.005+0000 7fe6c708dec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:26:32.072 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.005+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.080 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.015+0000 7fe6c708dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:26:32.081 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.015+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.093 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.027+0000 7fe6c708dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:26:32.093 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.027+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.036+0000 7fe6c708dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:26:32.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.036+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.112 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.046+0000 7fe6c708dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:26:32.112 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.046+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.051+0000 7fe6c708dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:32.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.051+0000 7fe6c708dec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:26:32.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.051+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.124 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.057+0000 7fe6c708dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:32.124 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.057+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.131 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.066+0000 7fe6c708dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:26:32.131 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.066+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.080+0000 7fe6c708dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:26:32.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.080+0000 7fe6c708dec0 20 zonegroup default 2019-12-09T09:26:32.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.080+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.153 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.087+0000 7fe6c708dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:32.153 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.087+0000 7fe6c708dec0 10 Cannot find current period zone using local zone 2019-12-09T09:26:32.153 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.087+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.163 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.097+0000 7fe6c708dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:32.163 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.097+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.176 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.110+0000 7fe6c708dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:26:32.191 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.110+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:32.192 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.122+0000 7fe6c708dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:26:32.192 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.122+0000 7fe6c708dec0 20 zone default found 2019-12-09T09:26:32.193 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.122+0000 7fe6c708dec0 20 started sync module instance, tier type = 2019-12-09T09:26:32.193 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.122+0000 7fe6c708dec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:26:32.563 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.498+0000 7fe6c708dec0 20 add_watcher() i=0 2019-12-09T09:26:32.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.498+0000 7fe6c708dec0 20 add_watcher() i=1 2019-12-09T09:26:32.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.498+0000 7fe6c708dec0 20 add_watcher() i=2 2019-12-09T09:26:32.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.498+0000 7fe6c708dec0 20 add_watcher() i=3 2019-12-09T09:26:32.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.498+0000 7fe6c708dec0 20 add_watcher() i=4 2019-12-09T09:26:32.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.498+0000 7fe6c708dec0 20 add_watcher() i=5 2019-12-09T09:26:32.568 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.503+0000 7fe6c708dec0 20 add_watcher() i=6 2019-12-09T09:26:32.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.543+0000 7fe6c708dec0 20 add_watcher() i=7 2019-12-09T09:26:32.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.543+0000 7fe6c708dec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:26:32.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.543+0000 7fe684fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:26:32.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.545+0000 7fe6b4ab8700 0 -- 172.21.15.35:0/154241919 >> v1:172.21.15.35:6789/0 conn(0x555dac1311c0 legacy=0x555dac1347b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T09:26:32.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:32.615+0000 7fe6b3ab6700 0 -- 172.21.15.35:0/2461810617 >> [v2:172.21.15.35:6808/9789,v1:172.21.15.35:6809/9789] conn(0x555dac149820 msgr2=0x555dac14bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.668+0000 7fe5f6ffd700 20 reqs_thread_entry: start 2019-12-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.668+0000 7fe5f4ff9700 20 reqs_thread_entry: start 2019-12-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.668+0000 7fe6c708dec0 20 init_complete bucket index max shards: 0 2019-12-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.669+0000 7fe5eaffd700 20 reqs_thread_entry: start 2019-12-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.669+0000 7fe6c708dec0 20 get_system_obj_state: rctx=0x7ffdc17d5e68 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x555dac1b62a0 s->prefetch_data=0 2019-12-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.669+0000 7fe6c708dec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-12-09T09:26:34.755 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.690+0000 7fe6c708dec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x16 2019-12-09T09:26:34.755 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.690+0000 7fe6c708dec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:26:34.756 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.690+0000 7fe6c708dec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:34.756 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.690+0000 7fe6c708dec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:34.756 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.690+0000 7fe6c708dec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T09:26:34.756 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.690+0000 7fe6c708dec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:34.769 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.703+0000 7fe6c708dec0 20 rados_obj.operate() r=0 bl.length=448 2019-12-09T09:26:34.769 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.703+0000 7fe6c708dec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x11 2019-12-09T09:26:34.769 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.703+0000 7fe6c708dec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:26:34.780 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.715+0000 7fe6c708dec0 10 removing key index: ZFXYDUXWOSUPFELXOEXO 2019-12-09T09:26:34.780 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.715+0000 7fe6c708dec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=170 2019-12-09T09:26:34.790 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:34.724+0000 7fe6a9ffb700 10 RGWWatcher::handle_notify() notify_id 575525617667 cookie 93860807181744 notifier 4490 bl.length()=170 2019-12-09T09:26:35.164 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:35.098+0000 7fe6c708dec0 10 removing email index: sysuser.client.0+test@test.test 2019-12-09T09:26:35.164 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:35.098+0000 7fe6c708dec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=182 2019-12-09T09:26:35.181 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:35.116+0000 7fe6a9ffb700 10 RGWWatcher::handle_notify() notify_id 575525617668 cookie 93860807181744 notifier 4490 bl.length()=182 2019-12-09T09:26:35.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:35.544+0000 7fe6c708dec0 10 removing user buckets index 2019-12-09T09:26:35.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:35.544+0000 7fe6c708dec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=173 2019-12-09T09:26:35.636 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:35.570+0000 7fe6a9ffb700 10 RGWWatcher::handle_notify() notify_id 575525617665 cookie 93860807152768 notifier 4490 bl.length()=173 2019-12-09T09:26:35.964 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:35.898+0000 7fe6c708dec0 10 removing user index: sysuser.client.0 2019-12-09T09:26:35.964 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:35.898+0000 7fe6c708dec0 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2019-12-09T09:26:35.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:35.898+0000 7fe6c708dec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=165 2019-12-09T09:26:35.985 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:35.919+0000 7fe6a9ffb700 10 RGWWatcher::handle_notify() notify_id 575525617666 cookie 93860807149296 notifier 4490 bl.length()=165 2019-12-09T09:26:36.566 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:36.500+0000 7fe6c708dec0 20 remove_watcher() i=0 2019-12-09T09:26:36.566 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:36.500+0000 7fe6c708dec0 2 removed watcher, disabling cache 2019-12-09T09:26:36.649 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:36.583+0000 7fe6c708dec0 20 remove_watcher() i=1 2019-12-09T09:26:36.683 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:36.617+0000 7fe6c708dec0 20 remove_watcher() i=2 2019-12-09T09:26:36.746 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:36.680+0000 7fe6c708dec0 20 remove_watcher() i=3 2019-12-09T09:26:36.813 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:36.747+0000 7fe6c708dec0 20 remove_watcher() i=4 2019-12-09T09:26:36.861 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:36.795+0000 7fe6c708dec0 20 remove_watcher() i=5 2019-12-09T09:26:36.906 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:36.840+0000 7fe6c708dec0 20 remove_watcher() i=6 2019-12-09T09:26:36.948 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:36.882+0000 7fe6c708dec0 20 remove_watcher() i=7 2019-12-09T09:26:36.958 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2019-12-09T09:26:37.083 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.017+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.108 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.042+0000 7f2d77c1fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:37.108 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.042+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.118 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.052+0000 7f2d77c1fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:37.118 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.052+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.124 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.058+0000 7f2d77c1fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:37.124 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.058+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.133 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.067+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:26:37.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.256+0000 7f2d77c1fec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:26:37.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.256+0000 7f2d77c1fec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:26:37.323 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.256+0000 7f2d77c1fec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:26:37.323 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.256+0000 7f2d77c1fec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:26:37.323 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.256+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.333 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.266+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:26:37.333 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.266+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.283+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:26:37.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.283+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.358 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.292+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:26:37.358 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.293+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.378 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.312+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:26:37.378 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.312+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.319+0000 7f2d77c1fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:37.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.319+0000 7f2d77c1fec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:26:37.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.319+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.391 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.325+0000 7f2d77c1fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:37.391 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.325+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.405 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.339+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:26:37.405 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.339+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.425 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.359+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:26:37.425 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.359+0000 7f2d77c1fec0 20 zonegroup default 2019-12-09T09:26:37.425 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.359+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.435 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.370+0000 7f2d77c1fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:37.436 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.370+0000 7f2d77c1fec0 10 Cannot find current period zone using local zone 2019-12-09T09:26:37.436 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.370+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.379+0000 7f2d77c1fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:26:37.445 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.379+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.460 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.395+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:26:37.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.395+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:37.486 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.420+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:26:37.486 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.420+0000 7f2d77c1fec0 20 zone default found 2019-12-09T09:26:37.486 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.420+0000 7f2d77c1fec0 20 started sync module instance, tier type = 2019-12-09T09:26:37.487 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.420+0000 7f2d77c1fec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:26:38.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.936+0000 7f2d77c1fec0 20 add_watcher() i=0 2019-12-09T09:26:38.003 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.936+0000 7f2d77c1fec0 20 add_watcher() i=1 2019-12-09T09:26:38.003 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.936+0000 7f2d77c1fec0 20 add_watcher() i=2 2019-12-09T09:26:38.003 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.936+0000 7f2d77c1fec0 20 add_watcher() i=3 2019-12-09T09:26:38.003 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.936+0000 7f2d77c1fec0 20 add_watcher() i=4 2019-12-09T09:26:38.003 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.936+0000 7f2d77c1fec0 20 add_watcher() i=5 2019-12-09T09:26:38.040 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.974+0000 7f2d77c1fec0 20 add_watcher() i=6 2019-12-09T09:26:38.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.990+0000 7f2d77c1fec0 20 add_watcher() i=7 2019-12-09T09:26:38.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.990+0000 7f2d77c1fec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:26:38.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:37.991+0000 7f2d357e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:26:38.078 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:38.012+0000 7f2d64e49700 0 -- 172.21.15.35:0/3079625807 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x5580803cf470 msgr2=0x5580803d5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:26:38.140 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:38.074+0000 7f2d6564a700 0 -- 172.21.15.35:0/152708313 >> [v2:172.21.15.22:6800/8484,v1:172.21.15.22:6801/8484] conn(0x7f2ca80224b0 msgr2=0x7f2ca80249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:26:39.859 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.793+0000 7f2ca77fe700 20 reqs_thread_entry: start 2019-12-09T09:26:39.860 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.793+0000 7f2ca57fa700 20 reqs_thread_entry: start 2019-12-09T09:26:39.860 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.794+0000 7f2d77c1fec0 20 init_complete bucket index max shards: 0 2019-12-09T09:26:39.860 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.794+0000 7f2c937fe700 20 reqs_thread_entry: start 2019-12-09T09:26:39.861 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.794+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9448 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x558080451400 s->prefetch_data=0 2019-12-09T09:26:39.861 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.794+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-12-09T09:26:39.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.822+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x16 2019-12-09T09:26:39.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.822+0000 7f2d77c1fec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:26:39.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.822+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:39.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.822+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:39.889 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.822+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T09:26:39.889 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.822+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:39.906 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.840+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=448 2019-12-09T09:26:39.906 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.840+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x11 2019-12-09T09:26:39.906 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.840+0000 7f2d77c1fec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:26:39.937 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.871+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-conf state=0x558080452d20 s->prefetch_data=0 2019-12-09T09:26:39.937 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.871+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2019-12-09T09:26:39.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.893+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x6 2019-12-09T09:26:39.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.893+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T09:26:39.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.893+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:39.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.893+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:39.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.893+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x1, cached=0x6) 2019-12-09T09:26:39.960 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.893+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:39.978 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.912+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=200 2019-12-09T09:26:39.978 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.912+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x1 2019-12-09T09:26:39.979 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.912+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T09:26:39.979 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.912+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-conf:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 state=0x558080457560 s->prefetch_data=0 2019-12-09T09:26:39.979 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.912+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 : miss 2019-12-09T09:26:39.998 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.932+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 info.flags=0x16 2019-12-09T09:26:39.998 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.932+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 to cache LRU end 2019-12-09T09:26:39.998 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.932+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:39.998 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.932+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:39.998 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.932+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:39.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.932+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:26:39.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.932+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:26:39.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.932+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:26:39.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.932+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:40.024 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.958+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=292 2019-12-09T09:26:40.024 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.958+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 info.flags=0x13 2019-12-09T09:26:40.025 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.958+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-conf:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 to cache LRU end 2019-12-09T09:26:40.025 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.958+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:40.025 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.958+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:40.025 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.958+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 2019-12-09T09:26:40.037 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.971+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-conf[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1]) start [] num_entries 1100 2019-12-09T09:26:40.055 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.989+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete[] 2019-12-09T09:26:40.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.989+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete[] 2019-12-09T09:26:40.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.989+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning[] 2019-12-09T09:26:40.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.989+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_simple[] 2019-12-09T09:26:40.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.989+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class[] 2019-12-09T09:26:40.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.989+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy[] 2019-12-09T09:26:40.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.989+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart[] 2019-12-09T09:26:40.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.989+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy[] 2019-12-09T09:26:40.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.989+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_small_obj_data[] 2019-12-09T09:26:40.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:39.989+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:40.068 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.002+0000 7f2d77c1fec0 10 manifest: total_size = 5949 2019-12-09T09:26:40.068 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.002+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.13 2019-12-09T09:26:40.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.002+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:40.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.002+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045a518 2019-12-09T09:26:40.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.002+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 2019-12-09T09:26:40.216 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.150+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:40.229 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.163+0000 7f2d77c1fec0 10 manifest: total_size = 3888 2019-12-09T09:26:40.230 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.163+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.30 2019-12-09T09:26:40.230 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.163+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:40.230 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.163+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045a518 2019-12-09T09:26:40.230 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.163+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 2019-12-09T09:26:40.411 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.344+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:40.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.353+0000 7f2d77c1fec0 10 manifest: total_size = 5689 2019-12-09T09:26:40.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.353+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.56 2019-12-09T09:26:40.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.353+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:40.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.353+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045a518 2019-12-09T09:26:40.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.353+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 2019-12-09T09:26:40.620 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.553+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:40.640 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.573+0000 7f2d77c1fec0 10 manifest: total_size = 5737 2019-12-09T09:26:40.640 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.573+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.72 2019-12-09T09:26:40.640 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.573+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:40.640 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.573+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045a518 2019-12-09T09:26:40.641 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.573+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 2019-12-09T09:26:40.816 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.749+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:40.832 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.764+0000 7f2d77c1fec0 10 manifest: total_size = 3922 2019-12-09T09:26:40.832 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.764+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.87 2019-12-09T09:26:40.832 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.764+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:40.833 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.764+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045a518 2019-12-09T09:26:40.833 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.764+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 2019-12-09T09:26:41.021 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.954+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:41.038 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.971+0000 7f2d77c1fec0 10 manifest: total_size = 3950 2019-12-09T09:26:41.038 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.971+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.120 2019-12-09T09:26:41.039 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.971+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:41.039 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.971+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045a518 2019-12-09T09:26:41.039 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:40.971+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 2019-12-09T09:26:41.220 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.152+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:41.243 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.176+0000 7f2d77c1fec0 10 manifest: total_size = 11054 2019-12-09T09:26:41.243 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.176+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.189 2019-12-09T09:26:41.243 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.176+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:41.244 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.176+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045a518 2019-12-09T09:26:41.244 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.176+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 2019-12-09T09:26:41.485 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.418+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:41.519 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.451+0000 7f2d77c1fec0 10 manifest: total_size = 3992 2019-12-09T09:26:41.519 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.451+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.246 2019-12-09T09:26:41.519 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.451+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:41.519 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.451+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045a518 2019-12-09T09:26:41.519 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.451+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 2019-12-09T09:26:41.641 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.574+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:41.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.588+0000 7f2d77c1fec0 10 manifest: total_size = 6776 2019-12-09T09:26:41.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.588+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.322 2019-12-09T09:26:41.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.588+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55808045a518 s->prefetch_data=0 2019-12-09T09:26:41.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.588+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045a518 2019-12-09T09:26:41.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.588+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 2019-12-09T09:26:41.865 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.799+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-conf[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1]) start _multipart_[] num_entries 1001 2019-12-09T09:26:41.865 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.799+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:26:41.961 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.895+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1])"; objs.size()=0; is_truncated=0 2019-12-09T09:26:42.043 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.977+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-conf state=0x558080459f70 s->prefetch_data=0 2019-12-09T09:26:42.043 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.977+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x16, cached=0x3) 2019-12-09T09:26:42.065 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.999+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x16 2019-12-09T09:26:42.065 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.999+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T09:26:42.065 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.999+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:42.066 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.999+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:42.066 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.999+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : hit (requested=0x11, cached=0x17) 2019-12-09T09:26:42.066 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.999+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-conf from cache 2019-12-09T09:26:42.066 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:41.999+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=162 2019-12-09T09:26:42.135 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.069+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617664 cookie 94010395558800 notifier 4496 bl.length()=162 2019-12-09T09:26:42.697 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.630+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.1 from cache 2019-12-09T09:26:42.697 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.630+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=219 2019-12-09T09:26:42.709 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.643+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617667 cookie 94010395565056 notifier 4496 bl.length()=219 2019-12-09T09:26:42.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.893+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x558080455270 s->prefetch_data=0 2019-12-09T09:26:42.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.893+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2019-12-09T09:26:42.995 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.929+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x6 2019-12-09T09:26:42.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.929+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T09:26:42.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.929+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:42.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.929+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:42.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.929+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:26:42.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.929+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:43.007 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.941+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=229 2019-12-09T09:26:43.007 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.941+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x1 2019-12-09T09:26:43.007 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.941+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T09:26:43.007 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.941+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2 state=0x558080459c70 s->prefetch_data=0 2019-12-09T09:26:43.008 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.941+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2 : miss 2019-12-09T09:26:43.028 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.962+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2 info.flags=0x16 2019-12-09T09:26:43.028 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.962+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2 to cache LRU end 2019-12-09T09:26:43.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.962+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:43.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.962+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:43.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.962+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:43.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.962+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:26:43.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.962+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:26:43.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.962+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:26:43.030 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.962+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:43.049 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.983+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=321 2019-12-09T09:26:43.049 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.983+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2 info.flags=0x13 2019-12-09T09:26:43.049 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.983+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2 to cache LRU end 2019-12-09T09:26:43.050 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.983+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:43.050 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.983+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:43.050 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:42.983+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2 2019-12-09T09:26:43.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.009+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-complete.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2]) start [] num_entries 1100 2019-12-09T09:26:43.140 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.074+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T09:26:43.140 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.074+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55808045a488 s->prefetch_data=0 2019-12-09T09:26:43.156 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.090+0000 7f2d77c1fec0 10 manifest: total_size = 1848 2019-12-09T09:26:43.156 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.090+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.20 2019-12-09T09:26:43.157 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.090+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55808045a488 s->prefetch_data=0 2019-12-09T09:26:43.157 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.090+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045a488 2019-12-09T09:26:43.157 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.090+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2 2019-12-09T09:26:43.348 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:26:43.348 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-12-09T09:26:43.348 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-12-09T09:26:43.348 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:26:43.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-12-09T09:26:43.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=515 2019-12-09T09:26:43.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:26:43.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=515 2019-12-09T09:26:43.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=1545 stripe_ofs=1545 part_ofs=1545 rule->part_size=303 2019-12-09T09:26:43.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=303 rules.size()=2 2019-12-09T09:26:43.350 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4195849 part_ofs=1545 rule->part_size=303 2019-12-09T09:26:43.350 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=1848 stripe_ofs=1848 part_ofs=1848 rule->part_size=303 2019-12-09T09:26:43.350 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.281+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.20 2019-12-09T09:26:43.419 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.353+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-complete.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2]) start _multipart_[] num_entries 1001 2019-12-09T09:26:43.419 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.353+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:26:43.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.367+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2])"; objs.size()=0; is_truncated=0 2019-12-09T09:26:43.505 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.438+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x558080459c70 s->prefetch_data=0 2019-12-09T09:26:43.505 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.438+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:26:43.521 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.454+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x16 2019-12-09T09:26:43.521 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.454+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T09:26:43.521 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.454+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:43.521 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.455+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:43.522 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.455+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:26:43.522 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.455+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2019-12-09T09:26:43.522 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.455+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=191 2019-12-09T09:26:43.587 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:43.521+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617664 cookie 94010395562144 notifier 4496 bl.length()=191 2019-12-09T09:26:44.320 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.253+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2 from cache 2019-12-09T09:26:44.320 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.253+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=248 2019-12-09T09:26:44.330 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.264+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617668 cookie 94010395583472 notifier 4496 bl.length()=248 2019-12-09T09:26:44.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.889+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x558080459c70 s->prefetch_data=0 2019-12-09T09:26:44.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.889+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2019-12-09T09:26:44.970 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.903+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x6 2019-12-09T09:26:44.970 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.903+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T09:26:44.970 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.903+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:44.970 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.903+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:44.970 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.903+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:26:44.970 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.903+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:44.981 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.914+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=236 2019-12-09T09:26:44.981 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.914+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x1 2019-12-09T09:26:44.981 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.914+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T09:26:44.981 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.914+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3 state=0x558080455270 s->prefetch_data=0 2019-12-09T09:26:44.981 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.914+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3 : miss 2019-12-09T09:26:45.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.934+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3 info.flags=0x16 2019-12-09T09:26:45.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.934+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3 to cache LRU end 2019-12-09T09:26:45.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.934+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:45.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.934+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:45.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.934+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:45.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.934+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:26:45.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.934+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:26:45.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.934+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:26:45.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.934+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:45.019 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.953+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=328 2019-12-09T09:26:45.020 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.953+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3 info.flags=0x13 2019-12-09T09:26:45.020 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.953+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3 to cache LRU end 2019-12-09T09:26:45.020 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.953+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:45.020 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.953+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:45.020 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.953+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3 2019-12-09T09:26:45.035 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.969+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-update-complete.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3]) start [] num_entries 1100 2019-12-09T09:26:45.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.990+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T09:26:45.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:44.990+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55808045a488 s->prefetch_data=0 2019-12-09T09:26:45.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.009+0000 7f2d77c1fec0 10 manifest: total_size = 886297 2019-12-09T09:26:45.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.009+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.40 2019-12-09T09:26:45.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.009+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55808045a488 s->prefetch_data=0 2019-12-09T09:26:45.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.009+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045a488 2019-12-09T09:26:45.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.009+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3 2019-12-09T09:26:45.195 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T09:26:45.195 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2019-12-09T09:26:45.195 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2019-12-09T09:26:45.195 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T09:26:45.195 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2019-12-09T09:26:45.196 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=246247 2019-12-09T09:26:45.196 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T09:26:45.196 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=246247 2019-12-09T09:26:45.196 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=738741 stripe_ofs=738741 part_ofs=738741 rule->part_size=147556 2019-12-09T09:26:45.196 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=147556 rules.size()=2 2019-12-09T09:26:45.196 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4933045 part_ofs=738741 rule->part_size=147556 2019-12-09T09:26:45.197 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=886297 stripe_ofs=886297 part_ofs=886297 rule->part_size=147556 2019-12-09T09:26:45.197 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.128+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.40 2019-12-09T09:26:45.272 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.206+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-update-complete.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3]) start _multipart_[] num_entries 1001 2019-12-09T09:26:45.273 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.206+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:26:45.286 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.219+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3])"; objs.size()=0; is_truncated=0 2019-12-09T09:26:45.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.320+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x558080460ec0 s->prefetch_data=0 2019-12-09T09:26:45.387 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.320+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:26:45.401 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.335+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x16 2019-12-09T09:26:45.402 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.335+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T09:26:45.402 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.335+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:45.402 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.335+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:45.402 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.335+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:26:45.402 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.335+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2019-12-09T09:26:45.402 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.335+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=198 2019-12-09T09:26:45.412 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.345+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617669 cookie 94010395565056 notifier 4496 bl.length()=198 2019-12-09T09:26:45.685 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.618+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3 from cache 2019-12-09T09:26:45.685 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.618+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=255 2019-12-09T09:26:45.697 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:45.631+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617669 cookie 94010395582368 notifier 4496 bl.length()=255 2019-12-09T09:26:46.237 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.170+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x558080459c70 s->prefetch_data=0 2019-12-09T09:26:46.237 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.170+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2019-12-09T09:26:46.251 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.185+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x6 2019-12-09T09:26:46.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.185+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T09:26:46.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.185+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:46.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.185+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:46.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.185+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:26:46.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.185+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:46.264 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.198+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=231 2019-12-09T09:26:46.265 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.198+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x1 2019-12-09T09:26:46.265 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.198+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T09:26:46.265 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.198+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 state=0x558080457850 s->prefetch_data=0 2019-12-09T09:26:46.265 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.198+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 : miss 2019-12-09T09:26:46.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.233+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 info.flags=0x16 2019-12-09T09:26:46.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.233+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 to cache LRU end 2019-12-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.233+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.233+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.233+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.233+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.233+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:26:46.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.233+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:26:46.301 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.233+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.249+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=323 2019-12-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.249+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 info.flags=0x13 2019-12-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.249+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 to cache LRU end 2019-12-09T09:26:46.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.249+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.249+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:46.317 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.249+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 2019-12-09T09:26:46.328 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.262+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-index-versioning.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4]) start [] num_entries 1100 2019-12-09T09:26:46.345 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.279+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo[sePiArn.nd1mnoldFezoFupSovJsOUj] 2019-12-09T09:26:46.346 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.279+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 2019-12-09T09:26:46.361 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.294+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x5580803a56b8 s->prefetch_data=0 2019-12-09T09:26:46.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.316+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to 7imvwcd8yn6nfb2h4rt2yjq1eph3kpp3 2019-12-09T09:26:46.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.316+0000 7f2d77c1fec0 20 get_obj_state_impl: setting s->olh_tag to vwdmagxrl31mct8v1zukf1t74lmc49rd 2019-12-09T09:26:46.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.316+0000 7f2d77c1fec0 20 bucket_index_guard_olh_op(): olh_state.olh_tag=vwdmagxrl31mct8v1zukf1t74lmc49rd 2019-12-09T09:26:46.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.392+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 2019-12-09T09:26:46.546 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.480+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 2019-12-09T09:26:46.547 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.480+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 2019-12-09T09:26:46.575 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.509+0000 7f2d77c1fec0 20 olh_log_entry: epoch=3 op=2 key=[] 2019-12-09T09:26:46.576 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.509+0000 7f2d77c1fec0 20 olh_log_entry: epoch=3 op=3 key=foo[sePiArn.nd1mnoldFezoFupSovJsOUj] 2019-12-09T09:26:46.576 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.509+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-multipart-index-versioning.1:_:sePiArn.nd1mnoldFezoFupSovJsOUj_foo state=0x558080467518 s->prefetch_data=0 2019-12-09T09:26:46.595 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.528+0000 7f2d77c1fec0 10 manifest: total_size = 74198 2019-12-09T09:26:46.595 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.528+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.55 2019-12-09T09:26:46.596 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.528+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-multipart-index-versioning.1:_:sePiArn.nd1mnoldFezoFupSovJsOUj_foo state=0x558080467518 s->prefetch_data=0 2019-12-09T09:26:46.596 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.528+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x558080467518 2019-12-09T09:26:46.596 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.528+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 2019-12-09T09:26:46.742 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T09:26:46.742 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=20613 2019-12-09T09:26:46.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-12-09T09:26:46.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T09:26:46.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4214917 part_ofs=20613 rule->part_size=20613 2019-12-09T09:26:46.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2019-12-09T09:26:46.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T09:26:46.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4235530 part_ofs=41226 rule->part_size=20613 2019-12-09T09:26:46.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2019-12-09T09:26:46.744 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2019-12-09T09:26:46.744 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4256143 part_ofs=61839 rule->part_size=12359 2019-12-09T09:26:46.744 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2019-12-09T09:26:46.744 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.675+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.55 2019-12-09T09:26:46.892 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.825+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 2019-12-09T09:26:46.892 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.825+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 2019-12-09T09:26:46.986 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.919+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 2019-12-09T09:26:47.049 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.982+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 2019-12-09T09:26:47.049 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.983+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-index-versioning.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4]) start _multipart_[] num_entries 1001 2019-12-09T09:26:47.050 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.983+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:26:47.061 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:46.995+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4])"; objs.size()=0; is_truncated=0 2019-12-09T09:26:47.147 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.081+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x5580803a76c0 s->prefetch_data=0 2019-12-09T09:26:47.148 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.081+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:26:47.160 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.093+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x16 2019-12-09T09:26:47.160 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.093+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T09:26:47.160 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.093+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:47.160 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.093+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:47.161 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.093+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:26:47.161 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.093+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2019-12-09T09:26:47.161 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.093+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=193 2019-12-09T09:26:47.168 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.101+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617670 cookie 94010395565056 notifier 4496 bl.length()=193 2019-12-09T09:26:47.323 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.257+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4 from cache 2019-12-09T09:26:47.323 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.257+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=250 2019-12-09T09:26:47.336 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.270+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617665 cookie 94010395588272 notifier 4496 bl.length()=250 2019-12-09T09:26:47.816 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.749+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x558080459c70 s->prefetch_data=0 2019-12-09T09:26:47.816 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.749+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2019-12-09T09:26:47.829 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.762+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x6 2019-12-09T09:26:47.829 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.762+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T09:26:47.829 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.762+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:47.829 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.762+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:47.829 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.762+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:26:47.829 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.762+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:47.839 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.772+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T09:26:47.839 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.772+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x1 2019-12-09T09:26:47.839 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.772+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T09:26:47.839 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.772+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5 state=0x558080457850 s->prefetch_data=0 2019-12-09T09:26:47.840 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.772+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5 : miss 2019-12-09T09:26:47.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.791+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5 info.flags=0x16 2019-12-09T09:26:47.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.791+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5 to cache LRU end 2019-12-09T09:26:47.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.791+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:47.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.791+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:47.859 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.791+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:47.859 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.791+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:26:47.859 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.791+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:26:47.859 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.791+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:26:47.859 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.791+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:47.877 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.810+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T09:26:47.877 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.810+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5 info.flags=0x13 2019-12-09T09:26:47.877 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.810+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5 to cache LRU end 2019-12-09T09:26:47.877 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.810+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:47.877 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.810+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:47.877 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.810+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5 2019-12-09T09:26:47.894 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.827+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-simple.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5]) start [] num_entries 1100 2019-12-09T09:26:47.919 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.852+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T09:26:47.919 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.852+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-multipart-simple.1:foo state=0x558080457a98 s->prefetch_data=0 2019-12-09T09:26:47.932 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.865+0000 7f2d77c1fec0 10 manifest: total_size = 1848 2019-12-09T09:26:47.933 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.865+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.71 2019-12-09T09:26:47.933 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.865+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-multipart-simple.1:foo state=0x558080457a98 s->prefetch_data=0 2019-12-09T09:26:47.933 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.865+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x558080457a98 2019-12-09T09:26:47.934 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:47.865+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5 2019-12-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-12-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-12-09T09:26:48.098 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-12-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=515 2019-12-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=515 2019-12-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=1545 stripe_ofs=1545 part_ofs=1545 rule->part_size=303 2019-12-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=303 rules.size()=2 2019-12-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4195849 part_ofs=1545 rule->part_size=303 2019-12-09T09:26:48.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=1848 stripe_ofs=1848 part_ofs=1848 rule->part_size=303 2019-12-09T09:26:48.100 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.031+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.71 2019-12-09T09:26:48.166 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.099+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-simple.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5]) start _multipart_[] num_entries 1001 2019-12-09T09:26:48.167 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.099+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:26:48.181 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.115+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5])"; objs.size()=0; is_truncated=0 2019-12-09T09:26:48.238 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.171+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x55808045b170 s->prefetch_data=0 2019-12-09T09:26:48.238 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.171+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:26:48.259 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.192+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x16 2019-12-09T09:26:48.259 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.192+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T09:26:48.260 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.192+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:48.260 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.192+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:48.260 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.192+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:26:48.260 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.192+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2019-12-09T09:26:48.260 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.192+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=183 2019-12-09T09:26:48.275 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.209+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617670 cookie 94010395582368 notifier 4496 bl.length()=183 2019-12-09T09:26:48.419 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.352+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5 from cache 2019-12-09T09:26:48.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.353+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=240 2019-12-09T09:26:48.442 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.376+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617665 cookie 94010395558800 notifier 4496 bl.length()=240 2019-12-09T09:26:48.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.860+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x558080459c70 s->prefetch_data=0 2019-12-09T09:26:48.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.860+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2019-12-09T09:26:48.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.898+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x6 2019-12-09T09:26:48.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.898+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T09:26:48.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.898+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:48.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.898+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:48.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.898+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:26:48.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.898+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:48.983 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.917+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=227 2019-12-09T09:26:48.984 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.917+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x1 2019-12-09T09:26:48.984 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.917+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T09:26:48.984 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.917+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 state=0x558080457850 s->prefetch_data=0 2019-12-09T09:26:48.984 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.917+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 : miss 2019-12-09T09:26:49.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.943+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 info.flags=0x16 2019-12-09T09:26:49.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.943+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 to cache LRU end 2019-12-09T09:26:49.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.943+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:49.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.943+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:49.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.943+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:49.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.943+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:26:49.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.943+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:26:49.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.943+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:26:49.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.943+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:49.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.966+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=319 2019-12-09T09:26:49.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.966+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 info.flags=0x13 2019-12-09T09:26:49.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.966+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 to cache LRU end 2019-12-09T09:26:49.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.966+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:49.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.966+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:49.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.966+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:49.053 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:48.983+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-copy.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7]) start [] num_entries 1100 2019-12-09T09:26:49.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T09:26:49.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-12-09T09:26:49.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-12-09T09:26:49.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-12-09T09:26:49.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T09:26:49.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-12-09T09:26:49.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-12-09T09:26:49.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-12-09T09:26:49.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T09:26:49.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-12-09T09:26:49.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-12-09T09:26:49.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-12-09T09:26:49.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.008+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:49.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.020+0000 7f2d77c1fec0 10 manifest: total_size = 5242880 2019-12-09T09:26:49.087 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.020+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.108 2019-12-09T09:26:49.087 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.020+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:49.087 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.020+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:49.087 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.020+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:49.194 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.127+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:49.194 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.127+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:26:49.194 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.127+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:49.194 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.127+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:26:49.194 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.127+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.108 2019-12-09T09:26:49.282 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.201+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:49.283 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.212+0000 7f2d77c1fec0 10 manifest: total_size = 5242880 2019-12-09T09:26:49.283 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.212+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.109 2019-12-09T09:26:49.283 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.212+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:49.283 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.212+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:49.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.212+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:49.381 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.314+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:49.381 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.314+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:26:49.381 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.314+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:49.382 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.314+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:26:49.382 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.314+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.109 2019-12-09T09:26:49.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.392+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:49.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.408+0000 7f2d77c1fec0 10 manifest: total_size = 5242880 2019-12-09T09:26:49.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.408+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _PubZ4ROdivGxPas0dy6OdH-WtOrxX6y 2019-12-09T09:26:49.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.408+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:49.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.408+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:49.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.408+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:49.602 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.535+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:49.602 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.535+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:26:49.602 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.535+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:49.602 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.535+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:26:49.602 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.535+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: _PubZ4ROdivGxPas0dy6OdH-WtOrxX6y 2019-12-09T09:26:49.660 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.593+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:49.677 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.610+0000 7f2d77c1fec0 10 manifest: total_size = 5242880 2019-12-09T09:26:49.677 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.610+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _HW0vY1jL3QytnL3rSqlKMo1bWIHZ_c_ 2019-12-09T09:26:49.677 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.610+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:49.677 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.610+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:49.677 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.610+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:49.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.735+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:49.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.735+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:26:49.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.735+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: _HW0vY1jL3QytnL3rSqlKMo1bWIHZ_c_ 2019-12-09T09:26:49.850 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.783+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:49.866 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.800+0000 7f2d77c1fec0 10 manifest: total_size = 5242880 2019-12-09T09:26:49.867 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.800+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.112 2019-12-09T09:26:49.867 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.800+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:49.867 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.800+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:49.867 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.800+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:49.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.929+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:49.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.929+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:26:49.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.929+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:49.997 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.929+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:26:49.997 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.929+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.112 2019-12-09T09:26:50.048 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.981+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.059 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.992+0000 7f2d77c1fec0 10 manifest: total_size = 5242880 2019-12-09T09:26:50.059 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.992+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _YFD1SJ0Iy7__-5-cf2JlaBATgCfcwka 2019-12-09T09:26:50.059 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.992+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.060 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.992+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:50.060 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:49.992+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:50.198 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.131+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:50.198 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.131+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:26:50.198 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.131+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:50.199 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.131+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:26:50.199 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.131+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: _YFD1SJ0Iy7__-5-cf2JlaBATgCfcwka 2019-12-09T09:26:50.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.217+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.334 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.267+0000 7f2d77c1fec0 10 manifest: total_size = 5242880 2019-12-09T09:26:50.334 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.267+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.114 2019-12-09T09:26:50.334 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.267+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.334 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.267+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:50.335 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.267+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:50.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.364+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:50.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.364+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:26:50.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.364+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:50.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.364+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:26:50.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.364+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.114 2019-12-09T09:26:50.481 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.414+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.423+0000 7f2d77c1fec0 10 manifest: total_size = 5242880 2019-12-09T09:26:50.491 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.423+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _0uP-qtn5iz4TxndlB9hXNZah1tBQh6P 2019-12-09T09:26:50.491 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.423+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.491 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.423+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:50.491 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.423+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:50.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.515+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:50.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.515+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:26:50.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.515+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: _0uP-qtn5iz4TxndlB9hXNZah1tBQh6P 2019-12-09T09:26:50.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.585+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.660 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.593+0000 7f2d77c1fec0 10 manifest: total_size = 2045 2019-12-09T09:26:50.660 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.593+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.116 2019-12-09T09:26:50.660 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.593+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.660 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.593+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:50.660 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.593+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:50.763 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.696+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.776 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.709+0000 7f2d77c1fec0 10 manifest: total_size = 2045 2019-12-09T09:26:50.776 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.709+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _A-d7hVRy2s1Q1kunFXYbfj-m3UCMzh- 2019-12-09T09:26:50.776 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.709+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.776 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.709+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:50.776 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.709+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:50.902 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.835+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:50.902 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.835+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=2045 stripe_ofs=2045 part_ofs=0 rule->part_size=0 2019-12-09T09:26:50.902 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.835+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: _A-d7hVRy2s1Q1kunFXYbfj-m3UCMzh- 2019-12-09T09:26:50.960 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.892+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.979 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.911+0000 7f2d77c1fec0 10 manifest: total_size = 2045 2019-12-09T09:26:50.980 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.911+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _yTkJfPzFsL4O6e-J2UN7YgOjcOEDNIS 2019-12-09T09:26:50.980 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.911+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:50.980 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.911+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:50.980 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:50.911+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:51.109 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.041+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:51.109 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.041+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=2045 stripe_ofs=2045 part_ofs=0 rule->part_size=0 2019-12-09T09:26:51.109 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.041+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: _yTkJfPzFsL4O6e-J2UN7YgOjcOEDNIS 2019-12-09T09:26:51.170 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.102+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:51.186 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.118+0000 7f2d77c1fec0 10 manifest: total_size = 2045 2019-12-09T09:26:51.187 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.118+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _NQfZ0RJBNt5OmFNm3V36Z6hxE51iedg 2019-12-09T09:26:51.187 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.118+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x5580804602f8 s->prefetch_data=0 2019-12-09T09:26:51.187 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.118+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804602f8 2019-12-09T09:26:51.187 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.118+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 2019-12-09T09:26:51.318 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.250+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-copy.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7]) start _multipart_[] num_entries 1001 2019-12-09T09:26:51.318 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.250+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:26:51.392 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.324+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7])"; objs.size()=0; is_truncated=0 2019-12-09T09:26:51.484 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.416+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x5580803a76c0 s->prefetch_data=0 2019-12-09T09:26:51.484 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.416+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:26:51.503 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.434+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x16 2019-12-09T09:26:51.503 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.434+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T09:26:51.503 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.434+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:51.503 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.434+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:51.503 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.434+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:26:51.503 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.435+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2019-12-09T09:26:51.504 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.435+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=189 2019-12-09T09:26:51.523 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.456+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617671 cookie 94010395565056 notifier 4496 bl.length()=189 2019-12-09T09:26:51.728 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.660+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7 from cache 2019-12-09T09:26:51.729 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.660+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=246 2019-12-09T09:26:51.750 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:51.682+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617666 cookie 94010395588272 notifier 4496 bl.length()=246 2019-12-09T09:26:52.333 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.264+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x5580803a76c0 s->prefetch_data=0 2019-12-09T09:26:52.333 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.264+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2019-12-09T09:26:52.347 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.279+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x6 2019-12-09T09:26:52.347 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.279+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T09:26:52.347 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.279+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:52.348 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.279+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:52.348 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.279+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:26:52.348 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.279+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:52.356 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.287+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=237 2019-12-09T09:26:52.356 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.287+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x1 2019-12-09T09:26:52.356 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.287+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T09:26:52.356 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.287+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 state=0x55808045aaa0 s->prefetch_data=0 2019-12-09T09:26:52.357 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.287+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 : miss 2019-12-09T09:26:52.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.317+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 info.flags=0x16 2019-12-09T09:26:52.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.317+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 to cache LRU end 2019-12-09T09:26:52.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.317+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:52.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.317+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:52.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.317+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:52.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.317+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:26:52.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.317+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:26:52.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.317+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:26:52.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.317+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:52.413 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.345+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=329 2019-12-09T09:26:52.413 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.345+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 info.flags=0x13 2019-12-09T09:26:52.413 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.345+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 to cache LRU end 2019-12-09T09:26:52.413 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.345+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:52.413 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.345+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:52.414 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.345+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:52.429 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.362+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9]) start [] num_entries 1100 2019-12-09T09:26:52.458 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T09:26:52.458 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-12-09T09:26:52.458 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-12-09T09:26:52.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-12-09T09:26:52.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T09:26:52.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-12-09T09:26:52.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-12-09T09:26:52.460 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-12-09T09:26:52.460 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T09:26:52.460 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-12-09T09:26:52.460 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-12-09T09:26:52.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-12-09T09:26:52.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.390+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55808045f2e8 s->prefetch_data=0 2019-12-09T09:26:52.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.407+0000 7f2d77c1fec0 10 manifest: total_size = 11534336 2019-12-09T09:26:52.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.407+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.224 2019-12-09T09:26:52.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.407+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55808045f2e8 s->prefetch_data=0 2019-12-09T09:26:52.476 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.407+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045f2e8 2019-12-09T09:26:52.476 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.407+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:52.621 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:52.621 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:52.621 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:52.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:52.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:52.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:26:52.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:52.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:26:52.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.553+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.224 2019-12-09T09:26:52.687 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.620+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:52.697 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.630+0000 7f2d77c1fec0 10 manifest: total_size = 11534336 2019-12-09T09:26:52.697 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.630+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.225 2019-12-09T09:26:52.698 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.630+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:52.698 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.630+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:52.698 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.630+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:52.800 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:52.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:52.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:52.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:52.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:52.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:52.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:52.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:52.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:52.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:52.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:52.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:52.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:26:52.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:52.803 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:26:52.803 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.733+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.225 2019-12-09T09:26:52.871 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.804+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:52.882 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.812+0000 7f2d77c1fec0 10 manifest: total_size = 11534336 2019-12-09T09:26:52.883 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.812+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _O7CcwLFjZorZ1nKZDh7jOaxefi3T8Rd 2019-12-09T09:26:52.883 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.812+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:52.883 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.812+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:52.883 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.812+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:52.986 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.919+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:52.987 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.919+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:26:52.987 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.919+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:52.987 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.919+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:26:52.987 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.919+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:52.987 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.919+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T09:26:52.988 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:52.919+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: _O7CcwLFjZorZ1nKZDh7jOaxefi3T8Rd 2019-12-09T09:26:53.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.019+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:53.101 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.034+0000 7f2d77c1fec0 10 manifest: total_size = 11534336 2019-12-09T09:26:53.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.034+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _MeqjdK6ZAkz9cutu_lg2TrrpHMIq3nX 2019-12-09T09:26:53.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.034+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:53.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.034+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:53.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.034+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:53.272 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.204+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:53.272 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.204+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:26:53.272 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.204+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:53.272 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.204+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T09:26:53.273 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.204+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: _MeqjdK6ZAkz9cutu_lg2TrrpHMIq3nX 2019-12-09T09:26:53.364 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.297+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:53.382 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.315+0000 7f2d77c1fec0 10 manifest: total_size = 11534336 2019-12-09T09:26:53.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.315+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.233 2019-12-09T09:26:53.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.315+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:53.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.315+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:53.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.315+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:53.497 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:53.497 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:53.497 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:53.497 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:53.497 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:53.498 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:53.498 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:53.498 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:53.498 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:53.498 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:53.498 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:53.498 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:53.499 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:26:53.499 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:53.499 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:26:53.499 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.429+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.233 2019-12-09T09:26:53.580 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.513+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:53.594 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.527+0000 7f2d77c1fec0 10 manifest: total_size = 11534336 2019-12-09T09:26:53.594 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.527+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _Xly0KU9dhfiB6N97XyLOKe-FZJwggVT 2019-12-09T09:26:53.594 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.527+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:53.595 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.527+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:53.595 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.527+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:53.724 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.657+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:53.725 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.657+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:26:53.725 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.657+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:53.725 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.657+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:26:53.725 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.657+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:53.725 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.657+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T09:26:53.725 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.657+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: _Xly0KU9dhfiB6N97XyLOKe-FZJwggVT 2019-12-09T09:26:53.848 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.781+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:53.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.790+0000 7f2d77c1fec0 10 manifest: total_size = 11534336 2019-12-09T09:26:53.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.790+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.235 2019-12-09T09:26:53.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.790+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:53.859 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.790+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:53.859 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.790+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:53.942 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:53.942 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:53.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:53.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:53.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:53.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:53.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:53.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:53.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:53.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:53.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:53.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:53.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:26:53.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:53.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:26:53.945 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.874+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.235 2019-12-09T09:26:54.015 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.948+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:54.028 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.961+0000 7f2d77c1fec0 10 manifest: total_size = 11534336 2019-12-09T09:26:54.028 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.961+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _Wr_T9xfJ8iCdEnymxS1_9iNk78pcoHp 2019-12-09T09:26:54.028 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.961+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:54.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.961+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:54.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:53.961+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:54.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.108+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:54.176 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.108+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:26:54.176 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.108+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:54.176 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.108+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T09:26:54.176 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.108+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _Wr_T9xfJ8iCdEnymxS1_9iNk78pcoHp 2019-12-09T09:26:54.222 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.155+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:54.231 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.164+0000 7f2d77c1fec0 10 manifest: total_size = 45360 2019-12-09T09:26:54.231 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.164+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.242 2019-12-09T09:26:54.231 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.164+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:54.231 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.164+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:54.232 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.164+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:54.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.258+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T09:26:54.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.258+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=20613 2019-12-09T09:26:54.326 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.258+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-12-09T09:26:54.326 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.258+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T09:26:54.326 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.258+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4214917 part_ofs=20613 rule->part_size=20613 2019-12-09T09:26:54.326 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.258+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=4134 2019-12-09T09:26:54.327 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.258+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=4134 rules.size()=2 2019-12-09T09:26:54.327 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.258+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4235530 part_ofs=41226 rule->part_size=4134 2019-12-09T09:26:54.327 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.258+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=45360 rule->part_size=4134 2019-12-09T09:26:54.327 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.258+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.242 2019-12-09T09:26:54.412 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.345+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:54.426 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.358+0000 7f2d77c1fec0 10 manifest: total_size = 45360 2019-12-09T09:26:54.426 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.358+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to _bSB6s2pvDxBc_Hn9b2Svio5PoxtdiKt 2019-12-09T09:26:54.426 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.358+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:54.426 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.358+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:54.427 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.358+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:54.561 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.493+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:54.561 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.493+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=0 rule->part_size=0 2019-12-09T09:26:54.561 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.493+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: _bSB6s2pvDxBc_Hn9b2Svio5PoxtdiKt 2019-12-09T09:26:54.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.563+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:54.650 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.582+0000 7f2d77c1fec0 10 manifest: total_size = 45360 2019-12-09T09:26:54.650 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.582+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to __KSkJPOzsJmV2gHP12D92Wv4IR8vdTy 2019-12-09T09:26:54.650 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.582+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:54.651 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.582+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:54.651 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.582+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:54.744 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:54.744 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.675+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=0 rule->part_size=0 2019-12-09T09:26:54.744 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.675+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: __KSkJPOzsJmV2gHP12D92Wv4IR8vdTy 2019-12-09T09:26:54.797 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.729+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:54.806 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.739+0000 7f2d77c1fec0 10 manifest: total_size = 45360 2019-12-09T09:26:54.807 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.739+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.245 2019-12-09T09:26:54.807 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.739+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:54.807 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.739+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:54.807 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.739+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 2019-12-09T09:26:54.933 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.865+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T09:26:54.934 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.865+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=20613 2019-12-09T09:26:54.934 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.865+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-12-09T09:26:54.934 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.865+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T09:26:54.934 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.865+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4214917 part_ofs=20613 rule->part_size=20613 2019-12-09T09:26:54.934 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.865+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=4134 2019-12-09T09:26:54.934 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.865+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=4134 rules.size()=2 2019-12-09T09:26:54.934 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.865+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4235530 part_ofs=41226 rule->part_size=4134 2019-12-09T09:26:54.935 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.865+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=45360 rule->part_size=4134 2019-12-09T09:26:54.935 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.865+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.245 2019-12-09T09:26:55.003 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.935+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9]) start _multipart_[] num_entries 1001 2019-12-09T09:26:55.003 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.935+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:26:55.016 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:54.949+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9])"; objs.size()=0; is_truncated=0 2019-12-09T09:26:55.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.023+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x55808045aaa0 s->prefetch_data=0 2019-12-09T09:26:55.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.023+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:26:55.103 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.036+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x16 2019-12-09T09:26:55.103 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.036+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T09:26:55.103 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.036+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:55.104 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.036+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:55.104 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.036+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:26:55.104 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.036+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2019-12-09T09:26:55.104 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.036+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=199 2019-12-09T09:26:55.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.050+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617666 cookie 94010395566688 notifier 4496 bl.length()=199 2019-12-09T09:26:55.764 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.697+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9 from cache 2019-12-09T09:26:55.765 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.697+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=256 2019-12-09T09:26:55.786 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:55.718+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617667 cookie 94010395602976 notifier 4496 bl.length()=256 2019-12-09T09:26:56.228 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.161+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x55808045aaa0 s->prefetch_data=0 2019-12-09T09:26:56.228 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.161+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2019-12-09T09:26:56.246 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.179+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x6 2019-12-09T09:26:56.246 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.179+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T09:26:56.246 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.179+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:56.247 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.179+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:56.247 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.179+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:26:56.247 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.179+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:56.263 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.196+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=232 2019-12-09T09:26:56.263 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.196+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x1 2019-12-09T09:26:56.264 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.196+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T09:26:56.264 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.196+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 state=0x5580803a76c0 s->prefetch_data=0 2019-12-09T09:26:56.264 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.196+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 : miss 2019-12-09T09:26:56.280 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.212+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 info.flags=0x16 2019-12-09T09:26:56.280 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.213+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 to cache LRU end 2019-12-09T09:26:56.280 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.213+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:56.280 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.213+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:56.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.213+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:56.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.213+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:26:56.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.213+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:26:56.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.213+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:26:56.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.213+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:56.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.232+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=324 2019-12-09T09:26:56.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.232+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 info.flags=0x13 2019-12-09T09:26:56.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.232+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 to cache LRU end 2019-12-09T09:26:56.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.232+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:56.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.232+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:56.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.232+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 2019-12-09T09:26:56.314 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.247+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8]) start [] num_entries 1100 2019-12-09T09:26:56.331 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.263+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T09:26:56.331 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.263+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T09:26:56.331 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.263+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T09:26:56.331 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.263+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:56.341 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.273+0000 7f2d77c1fec0 10 manifest: total_size = 11534336 2019-12-09T09:26:56.341 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.273+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.196 2019-12-09T09:26:56.341 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.273+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:56.341 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.273+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:56.342 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.273+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 2019-12-09T09:26:56.476 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:56.476 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:56.476 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:56.476 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:56.476 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:56.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:56.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:56.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:56.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:56.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:56.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:56.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:56.478 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:26:56.478 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:56.478 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:26:56.478 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.408+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.196 2019-12-09T09:26:56.516 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.449+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:56.558 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.458+0000 7f2d77c1fec0 10 manifest: total_size = 11534336 2019-12-09T09:26:56.558 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.458+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.203 2019-12-09T09:26:56.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.458+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:56.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.458+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:56.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.458+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 2019-12-09T09:26:56.679 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:56.679 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:56.679 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:56.680 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:56.680 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:26:56.680 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:56.680 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:56.680 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:56.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:56.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:26:56.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:26:56.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:56.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:26:56.682 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:26:56.682 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:26:56.682 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.611+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.203 2019-12-09T09:26:56.728 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.661+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:56.738 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.670+0000 7f2d77c1fec0 10 manifest: total_size = 541681 2019-12-09T09:26:56.738 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.670+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.210 2019-12-09T09:26:56.738 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.670+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:56.738 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.670+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:56.738 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.670+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 2019-12-09T09:26:56.840 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.772+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T09:26:56.840 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.772+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2019-12-09T09:26:56.840 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.772+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2019-12-09T09:26:56.840 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.772+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T09:26:56.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.772+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2019-12-09T09:26:56.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.772+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=49187 2019-12-09T09:26:56.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.772+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=49187 rules.size()=2 2019-12-09T09:26:56.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.772+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=49187 2019-12-09T09:26:56.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.772+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=541681 stripe_ofs=541681 part_ofs=541681 rule->part_size=49187 2019-12-09T09:26:56.842 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.772+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.210 2019-12-09T09:26:56.886 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.819+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8]) start _multipart_[] num_entries 1001 2019-12-09T09:26:56.886 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.819+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:26:56.905 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.831+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8])"; objs.size()=0; is_truncated=0 2019-12-09T09:26:56.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.876+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x55808045aaa0 s->prefetch_data=0 2019-12-09T09:26:56.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.876+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:26:56.968 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.900+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x16 2019-12-09T09:26:56.968 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.900+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T09:26:56.968 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.900+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:56.969 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.900+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:56.969 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.900+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:26:56.969 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.900+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2019-12-09T09:26:56.969 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.900+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=194 2019-12-09T09:26:56.989 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:56.922+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617672 cookie 94010395583472 notifier 4496 bl.length()=194 2019-12-09T09:26:57.186 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.119+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8 from cache 2019-12-09T09:26:57.186 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.119+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=251 2019-12-09T09:26:57.206 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.139+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617668 cookie 94010395602976 notifier 4496 bl.length()=251 2019-12-09T09:26:57.445 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.378+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x5580803a76c0 s->prefetch_data=0 2019-12-09T09:26:57.445 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.378+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2019-12-09T09:26:57.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.389+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x6 2019-12-09T09:26:57.457 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.389+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T09:26:57.457 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.389+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:57.457 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.389+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:57.457 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.389+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:26:57.457 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.389+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:57.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.397+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=222 2019-12-09T09:26:57.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.397+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x1 2019-12-09T09:26:57.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.398+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T09:26:57.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.398+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 state=0x55808045aaa0 s->prefetch_data=0 2019-12-09T09:26:57.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.398+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 : miss 2019-12-09T09:26:57.483 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.415+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 info.flags=0x16 2019-12-09T09:26:57.483 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.415+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 to cache LRU end 2019-12-09T09:26:57.483 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.415+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:57.483 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.415+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:57.483 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.415+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:57.484 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.415+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:26:57.484 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.415+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:26:57.484 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.415+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:26:57.484 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.415+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:57.512 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.444+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=314 2019-12-09T09:26:57.512 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.444+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 info.flags=0x13 2019-12-09T09:26:57.512 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.444+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 to cache LRU end 2019-12-09T09:26:57.512 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.444+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:57.512 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.444+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:57.513 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.444+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 2019-12-09T09:26:57.534 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.467+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6]) start [] num_entries 1100 2019-12-09T09:26:57.558 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.490+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T09:26:57.558 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.490+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T09:26:57.558 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.490+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T09:26:57.558 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.490+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:57.570 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.502+0000 7f2d77c1fec0 10 manifest: total_size = 10485760 2019-12-09T09:26:57.570 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.502+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.84 2019-12-09T09:26:57.570 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.502+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:57.571 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.502+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:57.571 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.502+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 2019-12-09T09:26:57.729 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.661+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:57.729 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.661+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:26:57.729 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.661+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:57.730 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.661+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:26:57.730 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.661+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:57.730 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.661+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-12-09T09:26:57.730 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.661+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.84 2019-12-09T09:26:57.788 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.721+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:57.797 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.729+0000 7f2d77c1fec0 10 manifest: total_size = 10485760 2019-12-09T09:26:57.797 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.729+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.85 2019-12-09T09:26:57.797 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.729+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:57.798 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.729+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:57.798 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.729+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 2019-12-09T09:26:57.902 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.835+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:57.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.835+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:26:57.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.835+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:57.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.835+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:26:57.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.835+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:26:57.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.835+0000 7f2d77c1fec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-12-09T09:26:57.904 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.835+0000 7f2d77c1fec0 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.85 2019-12-09T09:26:57.972 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.905+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:57.990 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.923+0000 7f2d77c1fec0 10 manifest: total_size = 53502 2019-12-09T09:26:57.991 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.923+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.86 2019-12-09T09:26:57.991 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.923+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:57.991 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.923+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:57.991 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:57.923+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 2019-12-09T09:26:58.144 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.076+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6]) start _multipart_[] num_entries 1001 2019-12-09T09:26:58.144 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.076+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:26:58.206 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.138+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6])"; objs.size()=0; is_truncated=0 2019-12-09T09:26:58.279 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.212+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x558080459f70 s->prefetch_data=0 2019-12-09T09:26:58.280 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.212+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:26:58.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.231+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x16 2019-12-09T09:26:58.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.231+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T09:26:58.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.231+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:58.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.231+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:58.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.231+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:26:58.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.231+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2019-12-09T09:26:58.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.231+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=184 2019-12-09T09:26:58.317 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.249+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617673 cookie 94010395565056 notifier 4496 bl.length()=184 2019-12-09T09:26:58.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.647+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6 from cache 2019-12-09T09:26:58.715 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.647+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=241 2019-12-09T09:26:58.732 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:58.664+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617671 cookie 94010395582368 notifier 4496 bl.length()=241 2019-12-09T09:26:59.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.048+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x5580803a76c0 s->prefetch_data=0 2019-12-09T09:26:59.116 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.048+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2019-12-09T09:26:59.154 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.086+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x6 2019-12-09T09:26:59.154 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.086+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T09:26:59.154 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.086+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:59.154 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.086+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:59.154 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.086+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:26:59.155 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.086+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:59.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.101+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T09:26:59.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.101+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x1 2019-12-09T09:26:59.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.101+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T09:26:59.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.101+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 state=0x55808045aaa0 s->prefetch_data=0 2019-12-09T09:26:59.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.101+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 : miss 2019-12-09T09:26:59.191 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.124+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 info.flags=0x16 2019-12-09T09:26:59.192 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.124+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 to cache LRU end 2019-12-09T09:26:59.192 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.124+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:59.192 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.124+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:59.192 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.124+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:26:59.192 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.124+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:26:59.193 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.124+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:26:59.193 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.124+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:26:59.193 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.124+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:26:59.226 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.158+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T09:26:59.226 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.158+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 info.flags=0x13 2019-12-09T09:26:59.226 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.158+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 to cache LRU end 2019-12-09T09:26:59.226 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.158+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:26:59.227 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.158+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:26:59.227 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.158+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 2019-12-09T09:26:59.245 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.178+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10]) start [] num_entries 1100 2019-12-09T09:26:59.280 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.213+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T09:26:59.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.213+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T09:26:59.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.213+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T09:26:59.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.213+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T09:26:59.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.213+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.1:__ state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:59.302 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.234+0000 7f2d77c1fec0 10 manifest: total_size = 1035 2019-12-09T09:26:59.302 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.234+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.305 2019-12-09T09:26:59.302 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.234+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.1:__ state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:59.302 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.234+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:59.302 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.234+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 2019-12-09T09:26:59.458 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.390+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:59.500 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.432+0000 7f2d77c1fec0 10 manifest: total_size = 1048576 2019-12-09T09:26:59.500 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.432+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.307 2019-12-09T09:26:59.500 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.432+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:59.500 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.432+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:59.500 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.432+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 2019-12-09T09:26:59.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.589+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.1:___ state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:59.675 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.607+0000 7f2d77c1fec0 10 manifest: total_size = 1048576 2019-12-09T09:26:59.675 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.607+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.306 2019-12-09T09:26:59.675 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.607+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.1:___ state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:59.675 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.607+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:59.675 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.607+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 2019-12-09T09:26:59.845 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.777+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.1:obj state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:59.881 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.814+0000 7f2d77c1fec0 10 manifest: total_size = 1035 2019-12-09T09:26:59.881 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.814+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.304 2019-12-09T09:26:59.882 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.814+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.1:obj state=0x5580804594a8 s->prefetch_data=0 2019-12-09T09:26:59.882 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.814+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x5580804594a8 2019-12-09T09:26:59.882 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.814+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 2019-12-09T09:27:00.058 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:26:59.991+0000 7f2d357e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:27:00.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.023+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10]) start _multipart_[] num_entries 1001 2019-12-09T09:27:00.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.023+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:27:00.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.107+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10])"; objs.size()=0; is_truncated=0 2019-12-09T09:27:00.243 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.176+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x5580803a76c0 s->prefetch_data=0 2019-12-09T09:27:00.244 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.176+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:27:00.275 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.206+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x16 2019-12-09T09:27:00.275 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.206+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T09:27:00.275 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.206+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:27:00.276 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.206+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:27:00.276 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.206+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:27:00.276 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.207+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2019-12-09T09:27:00.276 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.207+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T09:27:00.286 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:00.218+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617665 cookie 94010395562144 notifier 4496 bl.length()=181 2019-12-09T09:27:01.144 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.076+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.10 from cache 2019-12-09T09:27:01.144 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.076+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=239 2019-12-09T09:27:01.155 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.088+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617666 cookie 94010395558800 notifier 4496 bl.length()=239 2019-12-09T09:27:01.356 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.288+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x5580803a76c0 s->prefetch_data=0 2019-12-09T09:27:01.356 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.288+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2019-12-09T09:27:01.366 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.298+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x6 2019-12-09T09:27:01.366 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.298+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T09:27:01.366 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.298+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:27:01.366 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.298+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:27:01.366 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.298+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:27:01.367 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.298+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:01.378 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.310+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T09:27:01.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.310+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x1 2019-12-09T09:27:01.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.310+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T09:27:01.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.310+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 state=0x55808045aaa0 s->prefetch_data=0 2019-12-09T09:27:01.380 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.310+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 : miss 2019-12-09T09:27:01.397 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.329+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 info.flags=0x16 2019-12-09T09:27:01.397 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.329+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 to cache LRU end 2019-12-09T09:27:01.398 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.329+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:27:01.398 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.329+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:27:01.398 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.329+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:27:01.398 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.329+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:27:01.398 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.329+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:27:01.398 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.329+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:27:01.398 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.329+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:01.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.353+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T09:27:01.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.353+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 info.flags=0x13 2019-12-09T09:27:01.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.353+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 to cache LRU end 2019-12-09T09:27:01.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.353+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:27:01.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.353+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:27:01.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.353+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 2019-12-09T09:27:01.435 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.367+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.2[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11]) start [] num_entries 1100 2019-12-09T09:27:01.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.386+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T09:27:01.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.386+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T09:27:01.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.386+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T09:27:01.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.386+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T09:27:01.457 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.386+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:01.474 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.395+0000 7f2d77c1fec0 10 manifest: total_size = 24595 2019-12-09T09:27:01.474 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.395+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.312 2019-12-09T09:27:01.474 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.395+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:01.474 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.395+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045dff8 2019-12-09T09:27:01.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.395+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 2019-12-09T09:27:01.552 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.483+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:01.577 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.493+0000 7f2d77c1fec0 10 manifest: total_size = 524288 2019-12-09T09:27:01.577 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.493+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.314 2019-12-09T09:27:01.577 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.493+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:01.577 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.493+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045dff8 2019-12-09T09:27:01.577 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.493+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 2019-12-09T09:27:01.778 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.709+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:01.792 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.723+0000 7f2d77c1fec0 10 manifest: total_size = 62 2019-12-09T09:27:01.792 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.723+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.313 2019-12-09T09:27:01.792 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.723+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:01.793 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.723+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045dff8 2019-12-09T09:27:01.793 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.723+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 2019-12-09T09:27:01.929 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.860+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:01.940 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.871+0000 7f2d77c1fec0 10 manifest: total_size = 62 2019-12-09T09:27:01.940 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.871+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.311 2019-12-09T09:27:01.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.871+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:01.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.871+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045dff8 2019-12-09T09:27:01.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:01.871+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 2019-12-09T09:27:02.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.017+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.2[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11]) start _multipart_[] num_entries 1001 2019-12-09T09:27:02.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.017+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:27:02.199 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.130+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11])"; objs.size()=0; is_truncated=0 2019-12-09T09:27:02.259 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.190+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x558080459080 s->prefetch_data=0 2019-12-09T09:27:02.259 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.190+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:27:02.277 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.208+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x16 2019-12-09T09:27:02.277 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.208+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T09:27:02.277 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.208+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:27:02.277 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.208+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:27:02.278 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.208+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : hit (requested=0x11, cached=0x17) 2019-12-09T09:27:02.278 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.208+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2019-12-09T09:27:02.278 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.208+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T09:27:02.298 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.229+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617666 cookie 94010395562144 notifier 4496 bl.length()=181 2019-12-09T09:27:02.585 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.516+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.11 from cache 2019-12-09T09:27:02.585 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.516+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=239 2019-12-09T09:27:02.606 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:02.537+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617667 cookie 94010395558800 notifier 4496 bl.length()=239 2019-12-09T09:27:03.312 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.244+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x55808045aaa0 s->prefetch_data=0 2019-12-09T09:27:03.312 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.244+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2019-12-09T09:27:03.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.254+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x6 2019-12-09T09:27:03.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.254+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T09:27:03.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.254+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:27:03.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.254+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:27:03.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.254+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:27:03.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.254+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:03.330 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.262+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T09:27:03.330 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.262+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x1 2019-12-09T09:27:03.330 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.262+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T09:27:03.331 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.262+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de9210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 state=0x5580803a76c0 s->prefetch_data=0 2019-12-09T09:27:03.331 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.262+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 : miss 2019-12-09T09:27:03.350 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.282+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 info.flags=0x16 2019-12-09T09:27:03.351 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.282+0000 7f2d77c1fec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 to cache LRU end 2019-12-09T09:27:03.351 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.282+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:27:03.351 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.282+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:27:03.351 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.282+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:27:03.351 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.282+0000 7f2d77c1fec0 20 Read xattr: user.rgw.acl 2019-12-09T09:27:03.351 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.282+0000 7f2d77c1fec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:27:03.352 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.282+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:27:03.352 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.282+0000 7f2d77c1fec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:03.368 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.300+0000 7f2d77c1fec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T09:27:03.368 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.300+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 info.flags=0x13 2019-12-09T09:27:03.369 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.300+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 to cache LRU end 2019-12-09T09:27:03.369 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.300+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:27:03.369 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.300+0000 7f2d77c1fec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:27:03.369 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.300+0000 7f2d77c1fec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 2019-12-09T09:27:03.381 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.313+0000 7f2d77c1fec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.3[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12]) start [] num_entries 1100 2019-12-09T09:27:03.416 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.348+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T09:27:03.416 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.348+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T09:27:03.416 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.348+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T09:27:03.416 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.348+0000 7f2d77c1fec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T09:27:03.416 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.348+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:03.425 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.357+0000 7f2d77c1fec0 10 manifest: total_size = 0 2019-12-09T09:27:03.425 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.357+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.319 2019-12-09T09:27:03.425 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.357+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:03.426 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.357+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045dff8 2019-12-09T09:27:03.426 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.357+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 2019-12-09T09:27:03.538 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.470+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:03.585 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.517+0000 7f2d77c1fec0 10 manifest: total_size = 0 2019-12-09T09:27:03.585 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.517+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.321 2019-12-09T09:27:03.585 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.517+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:03.586 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.517+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045dff8 2019-12-09T09:27:03.586 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.517+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 2019-12-09T09:27:03.715 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.647+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:03.730 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.662+0000 7f2d77c1fec0 10 manifest: total_size = 0 2019-12-09T09:27:03.730 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.662+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.320 2019-12-09T09:27:03.730 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.662+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:03.730 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.662+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045dff8 2019-12-09T09:27:03.730 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.662+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 2019-12-09T09:27:03.913 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.845+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:03.926 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.858+0000 7f2d77c1fec0 10 manifest: total_size = 0 2019-12-09T09:27:03.926 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.858+0000 7f2d77c1fec0 20 get_obj_state: setting s->obj_tag to d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.318 2019-12-09T09:27:03.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.858+0000 7f2d77c1fec0 20 get_obj_state: rctx=0x7ffe61de9230 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55808045dff8 s->prefetch_data=0 2019-12-09T09:27:03.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.858+0000 7f2d77c1fec0 20 prepare_atomic_modification: state is not atomic. state=0x55808045dff8 2019-12-09T09:27:03.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.858+0000 7f2d77c1fec0 20 bucket index object: default.rgw.buckets.index:.dir.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 2019-12-09T09:27:04.048 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.979+0000 7f2d77c1fec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.3[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12]) start _multipart_[] num_entries 1001 2019-12-09T09:27:04.048 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:03.979+0000 7f2d77c1fec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:27:04.105 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.037+0000 7f2d77c1fec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12])"; objs.size()=0; is_truncated=0 2019-12-09T09:27:04.160 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.092+0000 7f2d77c1fec0 20 get_system_obj_state: rctx=0x7ffe61de8bc8 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x558080460510 s->prefetch_data=0 2019-12-09T09:27:04.161 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.092+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:27:04.177 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.109+0000 7f2d77c1fec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x16 2019-12-09T09:27:04.178 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.109+0000 7f2d77c1fec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T09:27:04.178 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.109+0000 7f2d77c1fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:27:04.178 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.109+0000 7f2d77c1fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:27:04.178 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.109+0000 7f2d77c1fec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : hit (requested=0x11, cached=0x17) 2019-12-09T09:27:04.179 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.109+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2019-12-09T09:27:04.179 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.109+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T09:27:04.189 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.120+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617667 cookie 94010395562144 notifier 4496 bl.length()=181 2019-12-09T09:27:04.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.254+0000 7f2d77c1fec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.12 from cache 2019-12-09T09:27:04.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.254+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=239 2019-12-09T09:27:04.353 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.285+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617668 cookie 94010395558800 notifier 4496 bl.length()=239 2019-12-09T09:27:04.866 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.798+0000 7f2d77c1fec0 10 removing key index: YAQERBREYAHHWLIDJILD 2019-12-09T09:27:04.866 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.798+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=170 2019-12-09T09:27:04.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:04.811+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617667 cookie 94010395566688 notifier 4496 bl.length()=170 2019-12-09T09:27:05.454 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:05.386+0000 7f2d77c1fec0 10 removing email index: ragweed.client.0+test@test.test 2019-12-09T09:27:05.454 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:05.386+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=182 2019-12-09T09:27:05.485 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:05.408+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617674 cookie 94010395565056 notifier 4496 bl.length()=182 2019-12-09T09:27:05.798 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:05.730+0000 7f2d77c1fec0 10 removing user buckets index 2019-12-09T09:27:05.799 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:05.730+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=173 2019-12-09T09:27:05.813 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:05.745+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617675 cookie 94010395583472 notifier 4496 bl.length()=173 2019-12-09T09:27:06.273 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.205+0000 7f2d77c1fec0 10 removing user index: ragweed.client.0 2019-12-09T09:27:06.274 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.205+0000 7f2d77c1fec0 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2019-12-09T09:27:06.274 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.205+0000 7f2d77c1fec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=165 2019-12-09T09:27:06.288 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.220+0000 7f2d5a7fc700 10 RGWWatcher::handle_notify() notify_id 575525617672 cookie 94010395582368 notifier 4496 bl.length()=165 2019-12-09T09:27:06.633 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.565+0000 7f2d77c1fec0 20 remove_watcher() i=0 2019-12-09T09:27:06.634 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.565+0000 7f2d77c1fec0 2 removed watcher, disabling cache 2019-12-09T09:27:06.676 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.608+0000 7f2d77c1fec0 20 remove_watcher() i=1 2019-12-09T09:27:06.706 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.638+0000 7f2d77c1fec0 20 remove_watcher() i=2 2019-12-09T09:27:06.774 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.706+0000 7f2d77c1fec0 20 remove_watcher() i=3 2019-12-09T09:27:06.822 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.754+0000 7f2d77c1fec0 20 remove_watcher() i=4 2019-12-09T09:27:06.871 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.802+0000 7f2d77c1fec0 20 remove_watcher() i=5 2019-12-09T09:27:06.932 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.863+0000 7f2d77c1fec0 20 remove_watcher() i=6 2019-12-09T09:27:06.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:06.931+0000 7f2d77c1fec0 20 remove_watcher() i=7 2019-12-09T09:27:07.011 INFO:tasks.ragweed:Removing ragweed... 2019-12-09T09:27:07.012 INFO:teuthology.orchestra.run.smithi035:> rm -rf /home/ubuntu/cephtest/ragweed 2019-12-09T09:27:07.148 INFO:teuthology.run_tasks:Running task s3tests-java... 2019-12-09T09:27:07.161 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2019-12-09T09:27:07.162 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2019-12-09T09:27:07.162 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2019-12-09T09:27:07.162 INFO:teuthology.orchestra.run.smithi035:> git clone -b master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2019-12-09T09:27:07.211 INFO:teuthology.orchestra.run.smithi035.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2019-12-09T09:27:10.977 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2019-12-09T09:27:10.977 INFO:teuthology.orchestra.run.smithi035:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2019-12-09T09:27:13.680 INFO:teuthology.orchestra.run.smithi035.stdout: Gradle will be installed under /opt/gradle 2019-12-09T09:27:13.740 INFO:teuthology.orchestra.run.smithi035.stderr:--2019-12-09 09:27:13-- https://services.gradle.org/distributions/gradle-4.7-bin.zip 2019-12-09T09:27:13.746 INFO:teuthology.orchestra.run.smithi035.stderr:Resolving services.gradle.org (services.gradle.org)... 104.18.190.9, 104.18.191.9, 2606:4700::6812:be09, ... 2019-12-09T09:27:13.754 INFO:teuthology.orchestra.run.smithi035.stderr:Connecting to services.gradle.org (services.gradle.org)|104.18.190.9|:443... connected. 2019-12-09T09:27:13.791 INFO:teuthology.orchestra.run.smithi035.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2019-12-09T09:27:13.792 INFO:teuthology.orchestra.run.smithi035.stderr:Location: https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip [following] 2019-12-09T09:27:13.792 INFO:teuthology.orchestra.run.smithi035.stderr:--2019-12-09 09:27:13-- https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip 2019-12-09T09:27:13.792 INFO:teuthology.orchestra.run.smithi035.stderr:Resolving downloads.gradle-dn.com (downloads.gradle-dn.com)... 104.17.160.20, 104.17.159.20, 2606:4700::6811:9f14, ... 2019-12-09T09:27:13.800 INFO:teuthology.orchestra.run.smithi035.stderr:Connecting to downloads.gradle-dn.com (downloads.gradle-dn.com)|104.17.160.20|:443... connected. 2019-12-09T09:27:13.838 INFO:teuthology.orchestra.run.smithi035.stderr:HTTP request sent, awaiting response... 200 OK 2019-12-09T09:27:13.838 INFO:teuthology.orchestra.run.smithi035.stderr:Length: 75323957 (72M) [application/zip] 2019-12-09T09:27:13.838 INFO:teuthology.orchestra.run.smithi035.stderr:Saving to: ‘gradle-4.7-bin.zip’ 2019-12-09T09:27:13.839 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T09:27:13.845 INFO:teuthology.orchestra.run.smithi035.stderr: 0K .......... .......... .......... .......... .......... 0% 5.96M 12s 2019-12-09T09:27:13.848 INFO:teuthology.orchestra.run.smithi035.stderr: 50K .......... .......... .......... .......... .......... 0% 56.8M 7s 2019-12-09T09:27:13.853 INFO:teuthology.orchestra.run.smithi035.stderr: 100K .......... .......... .......... .......... .......... 0% 8.54M 7s 2019-12-09T09:27:13.855 INFO:teuthology.orchestra.run.smithi035.stderr: 150K .......... .......... .......... .......... .......... 0% 33.2M 6s 2019-12-09T09:27:13.855 INFO:teuthology.orchestra.run.smithi035.stderr: 200K .......... .......... .......... .......... .......... 0% 126M 5s 2019-12-09T09:27:13.857 INFO:teuthology.orchestra.run.smithi035.stderr: 250K .......... .......... .......... .......... .......... 0% 126M 4s 2019-12-09T09:27:13.857 INFO:teuthology.orchestra.run.smithi035.stderr: 300K .......... .......... .......... .......... .......... 0% 178M 4s 2019-12-09T09:27:13.860 INFO:teuthology.orchestra.run.smithi035.stderr: 350K .......... .......... .......... .......... .......... 0% 9.29M 4s 2019-12-09T09:27:13.862 INFO:teuthology.orchestra.run.smithi035.stderr: 400K .......... .......... .......... .......... .......... 0% 125M 4s 2019-12-09T09:27:13.862 INFO:teuthology.orchestra.run.smithi035.stderr: 450K .......... .......... .......... .......... .......... 0% 107M 3s 2019-12-09T09:27:13.865 INFO:teuthology.orchestra.run.smithi035.stderr: 500K .......... .......... .......... .......... .......... 0% 110M 3s 2019-12-09T09:27:13.866 INFO:teuthology.orchestra.run.smithi035.stderr: 550K .......... .......... .......... .......... .......... 0% 119M 3s 2019-12-09T09:27:13.866 INFO:teuthology.orchestra.run.smithi035.stderr: 600K .......... .......... .......... .......... .......... 0% 156M 3s 2019-12-09T09:27:13.866 INFO:teuthology.orchestra.run.smithi035.stderr: 650K .......... .......... .......... .......... .......... 0% 127M 3s 2019-12-09T09:27:13.866 INFO:teuthology.orchestra.run.smithi035.stderr: 700K .......... .......... .......... .......... .......... 1% 115M 2s 2019-12-09T09:27:13.866 INFO:teuthology.orchestra.run.smithi035.stderr: 750K .......... .......... .......... .......... .......... 1% 114M 2s 2019-12-09T09:27:13.867 INFO:teuthology.orchestra.run.smithi035.stderr: 800K .......... .......... .......... .......... .......... 1% 138M 2s 2019-12-09T09:27:13.868 INFO:teuthology.orchestra.run.smithi035.stderr: 850K .......... .......... .......... .......... .......... 1% 11.7M 2s 2019-12-09T09:27:13.871 INFO:teuthology.orchestra.run.smithi035.stderr: 900K .......... .......... .......... .......... .......... 1% 87.8M 2s 2019-12-09T09:27:13.871 INFO:teuthology.orchestra.run.smithi035.stderr: 950K .......... .......... .......... .......... .......... 1% 126M 2s 2019-12-09T09:27:13.872 INFO:teuthology.orchestra.run.smithi035.stderr: 1000K .......... .......... .......... .......... .......... 1% 145M 2s 2019-12-09T09:27:13.872 INFO:teuthology.orchestra.run.smithi035.stderr: 1050K .......... .......... .......... .......... .......... 1% 119M 2s 2019-12-09T09:27:13.881 INFO:teuthology.orchestra.run.smithi035.stderr: 1100K .......... .......... .......... .......... .......... 1% 105M 2s 2019-12-09T09:27:13.881 INFO:teuthology.orchestra.run.smithi035.stderr: 1150K .......... .......... .......... .......... .......... 1% 155M 2s 2019-12-09T09:27:13.881 INFO:teuthology.orchestra.run.smithi035.stderr: 1200K .......... .......... .......... .......... .......... 1% 188M 2s 2019-12-09T09:27:13.881 INFO:teuthology.orchestra.run.smithi035.stderr: 1250K .......... .......... .......... .......... .......... 1% 169M 2s 2019-12-09T09:27:13.881 INFO:teuthology.orchestra.run.smithi035.stderr: 1300K .......... .......... .......... .......... .......... 1% 98.5M 2s 2019-12-09T09:27:13.882 INFO:teuthology.orchestra.run.smithi035.stderr: 1350K .......... .......... .......... .......... .......... 1% 209M 2s 2019-12-09T09:27:13.882 INFO:teuthology.orchestra.run.smithi035.stderr: 1400K .......... .......... .......... .......... .......... 1% 235M 2s 2019-12-09T09:27:13.882 INFO:teuthology.orchestra.run.smithi035.stderr: 1450K .......... .......... .......... .......... .......... 2% 195M 2s 2019-12-09T09:27:13.882 INFO:teuthology.orchestra.run.smithi035.stderr: 1500K .......... .......... .......... .......... .......... 2% 152M 2s 2019-12-09T09:27:13.882 INFO:teuthology.orchestra.run.smithi035.stderr: 1550K .......... .......... .......... .......... .......... 2% 147M 2s 2019-12-09T09:27:14.979 INFO:teuthology.orchestra.run.smithi035.stderr: 1600K .......... .......... .......... .......... .......... 2% 146M 2s 2019-12-09T09:27:14.980 INFO:teuthology.orchestra.run.smithi035.stderr: 1650K .......... .......... .......... .......... .......... 2% 158M 2s 2019-12-09T09:27:14.980 INFO:teuthology.orchestra.run.smithi035.stderr: 1700K .......... .......... .......... .......... .......... 2% 32.2M 2s 2019-12-09T09:27:14.980 INFO:teuthology.orchestra.run.smithi035.stderr: 1750K .......... .......... .......... .......... .......... 2% 91.3M 2s 2019-12-09T09:27:14.980 INFO:teuthology.orchestra.run.smithi035.stderr: 1800K .......... .......... .......... .......... .......... 2% 191M 1s 2019-12-09T09:27:14.980 INFO:teuthology.orchestra.run.smithi035.stderr: 1850K .......... .......... .......... .......... .......... 2% 201M 1s 2019-12-09T09:27:14.980 INFO:teuthology.orchestra.run.smithi035.stderr: 1900K .......... .......... .......... .......... .......... 2% 159M 1s 2019-12-09T09:27:14.981 INFO:teuthology.orchestra.run.smithi035.stderr: 1950K .......... .......... .......... .......... .......... 2% 138M 1s 2019-12-09T09:27:14.981 INFO:teuthology.orchestra.run.smithi035.stderr: 2000K .......... .......... .......... .......... .......... 2% 205M 1s 2019-12-09T09:27:14.981 INFO:teuthology.orchestra.run.smithi035.stderr: 2050K .......... .......... .......... .......... .......... 2% 178M 1s 2019-12-09T09:27:14.981 INFO:teuthology.orchestra.run.smithi035.stderr: 2100K .......... .......... .......... .......... .......... 2% 257M 1s 2019-12-09T09:27:14.981 INFO:teuthology.orchestra.run.smithi035.stderr: 2150K .......... .......... .......... .......... .......... 2% 152M 1s 2019-12-09T09:27:14.981 INFO:teuthology.orchestra.run.smithi035.stderr: 2200K .......... .......... .......... .......... .......... 3% 183M 1s 2019-12-09T09:27:14.981 INFO:teuthology.orchestra.run.smithi035.stderr: 2250K .......... .......... .......... .......... .......... 3% 290M 1s 2019-12-09T09:27:14.982 INFO:teuthology.orchestra.run.smithi035.stderr: 2300K .......... .......... .......... .......... .......... 3% 169M 1s 2019-12-09T09:27:14.982 INFO:teuthology.orchestra.run.smithi035.stderr: 2350K .......... .......... .......... .......... .......... 3% 172M 1s 2019-12-09T09:27:14.982 INFO:teuthology.orchestra.run.smithi035.stderr: 2400K .......... .......... .......... .......... .......... 3% 153M 1s 2019-12-09T09:27:14.982 INFO:teuthology.orchestra.run.smithi035.stderr: 2450K .......... .......... .......... .......... .......... 3% 138M 1s 2019-12-09T09:27:14.982 INFO:teuthology.orchestra.run.smithi035.stderr: 2500K .......... .......... .......... .......... .......... 3% 155M 1s 2019-12-09T09:27:14.982 INFO:teuthology.orchestra.run.smithi035.stderr: 2550K .......... .......... .......... .......... .......... 3% 136M 1s 2019-12-09T09:27:14.983 INFO:teuthology.orchestra.run.smithi035.stderr: 2600K .......... .......... .......... .......... .......... 3% 157M 1s 2019-12-09T09:27:14.983 INFO:teuthology.orchestra.run.smithi035.stderr: 2650K .......... .......... .......... .......... .......... 3% 148M 1s 2019-12-09T09:27:14.983 INFO:teuthology.orchestra.run.smithi035.stderr: 2700K .......... .......... .......... .......... .......... 3% 156M 1s 2019-12-09T09:27:14.983 INFO:teuthology.orchestra.run.smithi035.stderr: 2750K .......... .......... .......... .......... .......... 3% 127M 1s 2019-12-09T09:27:14.983 INFO:teuthology.orchestra.run.smithi035.stderr: 2800K .......... .......... .......... .......... .......... 3% 131M 1s 2019-12-09T09:27:14.983 INFO:teuthology.orchestra.run.smithi035.stderr: 2850K .......... .......... .......... .......... .......... 3% 122M 1s 2019-12-09T09:27:14.983 INFO:teuthology.orchestra.run.smithi035.stderr: 2900K .......... .......... .......... .......... .......... 4% 117M 1s 2019-12-09T09:27:14.984 INFO:teuthology.orchestra.run.smithi035.stderr: 2950K .......... .......... .......... .......... .......... 4% 124M 1s 2019-12-09T09:27:14.984 INFO:teuthology.orchestra.run.smithi035.stderr: 3000K .......... .......... .......... .......... .......... 4% 150M 1s 2019-12-09T09:27:14.984 INFO:teuthology.orchestra.run.smithi035.stderr: 3050K .......... .......... .......... .......... .......... 4% 162M 1s 2019-12-09T09:27:14.984 INFO:teuthology.orchestra.run.smithi035.stderr: 3100K .......... .......... .......... .......... .......... 4% 167M 1s 2019-12-09T09:27:14.984 INFO:teuthology.orchestra.run.smithi035.stderr: 3150K .......... .......... .......... .......... .......... 4% 140M 1s 2019-12-09T09:27:14.984 INFO:teuthology.orchestra.run.smithi035.stderr: 3200K .......... .......... .......... .......... .......... 4% 154M 1s 2019-12-09T09:27:14.984 INFO:teuthology.orchestra.run.smithi035.stderr: 3250K .......... .......... .......... .......... .......... 4% 170M 1s 2019-12-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi035.stderr: 3300K .......... .......... .......... .......... .......... 4% 153M 1s 2019-12-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi035.stderr: 3350K .......... .......... .......... .......... .......... 4% 133M 1s 2019-12-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi035.stderr: 3400K .......... .......... .......... .......... .......... 4% 134M 1s 2019-12-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi035.stderr: 3450K .......... .......... .......... .......... .......... 4% 132M 1s 2019-12-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi035.stderr: 3500K .......... .......... .......... .......... .......... 4% 131M 1s 2019-12-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi035.stderr: 3550K .......... .......... .......... .......... .......... 4% 119M 1s 2019-12-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi035.stderr: 3600K .......... .......... .......... .......... .......... 4% 127M 1s 2019-12-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi035.stderr: 3650K .......... .......... .......... .......... .......... 5% 129M 1s 2019-12-09T09:27:14.986 INFO:teuthology.orchestra.run.smithi035.stderr: 3700K .......... .......... .......... .......... .......... 5% 110M 1s 2019-12-09T09:27:14.986 INFO:teuthology.orchestra.run.smithi035.stderr: 3750K .......... .......... .......... .......... .......... 5% 136M 1s 2019-12-09T09:27:14.986 INFO:teuthology.orchestra.run.smithi035.stderr: 3800K .......... .......... .......... .......... .......... 5% 172M 1s 2019-12-09T09:27:14.986 INFO:teuthology.orchestra.run.smithi035.stderr: 3850K .......... .......... .......... .......... .......... 5% 168M 1s 2019-12-09T09:27:14.986 INFO:teuthology.orchestra.run.smithi035.stderr: 3900K .......... .......... .......... .......... .......... 5% 142M 1s 2019-12-09T09:27:14.986 INFO:teuthology.orchestra.run.smithi035.stderr: 3950K .......... .......... .......... .......... .......... 5% 97.0M 1s 2019-12-09T09:27:14.986 INFO:teuthology.orchestra.run.smithi035.stderr: 4000K .......... .......... .......... .......... .......... 5% 120M 1s 2019-12-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi035.stderr: 4050K .......... .......... .......... .......... .......... 5% 136M 1s 2019-12-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi035.stderr: 4100K .......... .......... .......... .......... .......... 5% 137M 1s 2019-12-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi035.stderr: 4150K .......... .......... .......... .......... .......... 5% 122M 1s 2019-12-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi035.stderr: 4200K .......... .......... .......... .......... .......... 5% 146M 1s 2019-12-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi035.stderr: 4250K .......... .......... .......... .......... .......... 5% 142M 1s 2019-12-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi035.stderr: 4300K .......... .......... .......... .......... .......... 5% 149M 1s 2019-12-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi035.stderr: 4350K .......... .......... .......... .......... .......... 5% 137M 1s 2019-12-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi035.stderr: 4400K .......... .......... .......... .......... .......... 6% 123M 1s 2019-12-09T09:27:14.988 INFO:teuthology.orchestra.run.smithi035.stderr: 4450K .......... .......... .......... .......... .......... 6% 130M 1s 2019-12-09T09:27:14.988 INFO:teuthology.orchestra.run.smithi035.stderr: 4500K .......... .......... .......... .......... .......... 6% 149M 1s 2019-12-09T09:27:14.988 INFO:teuthology.orchestra.run.smithi035.stderr: 4550K .......... .......... .......... .......... .......... 6% 142M 1s 2019-12-09T09:27:14.988 INFO:teuthology.orchestra.run.smithi035.stderr: 4600K .......... .......... .......... .......... .......... 6% 137M 1s 2019-12-09T09:27:14.988 INFO:teuthology.orchestra.run.smithi035.stderr: 4650K .......... .......... .......... .......... .......... 6% 163M 1s 2019-12-09T09:27:14.988 INFO:teuthology.orchestra.run.smithi035.stderr: 4700K .......... .......... .......... .......... .......... 6% 149M 1s 2019-12-09T09:27:14.988 INFO:teuthology.orchestra.run.smithi035.stderr: 4750K .......... .......... .......... .......... .......... 6% 117M 1s 2019-12-09T09:27:14.989 INFO:teuthology.orchestra.run.smithi035.stderr: 4800K .......... .......... .......... .......... .......... 6% 140M 1s 2019-12-09T09:27:14.989 INFO:teuthology.orchestra.run.smithi035.stderr: 4850K .......... .......... .......... .......... .......... 6% 131M 1s 2019-12-09T09:27:14.989 INFO:teuthology.orchestra.run.smithi035.stderr: 4900K .......... .......... .......... .......... .......... 6% 129M 1s 2019-12-09T09:27:14.989 INFO:teuthology.orchestra.run.smithi035.stderr: 4950K .......... .......... .......... .......... .......... 6% 129M 1s 2019-12-09T09:27:14.989 INFO:teuthology.orchestra.run.smithi035.stderr: 5000K .......... .......... .......... .......... .......... 6% 140M 1s 2019-12-09T09:27:14.989 INFO:teuthology.orchestra.run.smithi035.stderr: 5050K .......... .......... .......... .......... .......... 6% 136M 1s 2019-12-09T09:27:14.989 INFO:teuthology.orchestra.run.smithi035.stderr: 5100K .......... .......... .......... .......... .......... 7% 175M 1s 2019-12-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi035.stderr: 5150K .......... .......... .......... .......... .......... 7% 133M 1s 2019-12-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi035.stderr: 5200K .......... .......... .......... .......... .......... 7% 146M 1s 2019-12-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi035.stderr: 5250K .......... .......... .......... .......... .......... 7% 180M 1s 2019-12-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi035.stderr: 5300K .......... .......... .......... .......... .......... 7% 147M 1s 2019-12-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi035.stderr: 5350K .......... .......... .......... .......... .......... 7% 128M 1s 2019-12-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi035.stderr: 5400K .......... .......... .......... .......... .......... 7% 161M 1s 2019-12-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi035.stderr: 5450K .......... .......... .......... .......... .......... 7% 160M 1s 2019-12-09T09:27:14.991 INFO:teuthology.orchestra.run.smithi035.stderr: 5500K .......... .......... .......... .......... .......... 7% 161M 1s 2019-12-09T09:27:14.991 INFO:teuthology.orchestra.run.smithi035.stderr: 5550K .......... .......... .......... .......... .......... 7% 151M 1s 2019-12-09T09:27:14.991 INFO:teuthology.orchestra.run.smithi035.stderr: 5600K .......... .......... .......... .......... .......... 7% 167M 1s 2019-12-09T09:27:14.991 INFO:teuthology.orchestra.run.smithi035.stderr: 5650K .......... .......... .......... .......... .......... 7% 184M 1s 2019-12-09T09:27:14.991 INFO:teuthology.orchestra.run.smithi035.stderr: 5700K .......... .......... .......... .......... .......... 7% 175M 1s 2019-12-09T09:27:14.991 INFO:teuthology.orchestra.run.smithi035.stderr: 5750K .......... .......... .......... .......... .......... 7% 141M 1s 2019-12-09T09:27:14.992 INFO:teuthology.orchestra.run.smithi035.stderr: 5800K .......... .......... .......... .......... .......... 7% 153M 1s 2019-12-09T09:27:14.992 INFO:teuthology.orchestra.run.smithi035.stderr: 5850K .......... .......... .......... .......... .......... 8% 171M 1s 2019-12-09T09:27:14.992 INFO:teuthology.orchestra.run.smithi035.stderr: 5900K .......... .......... .......... .......... .......... 8% 172M 1s 2019-12-09T09:27:14.992 INFO:teuthology.orchestra.run.smithi035.stderr: 5950K .......... .......... .......... .......... .......... 8% 129M 1s 2019-12-09T09:27:14.992 INFO:teuthology.orchestra.run.smithi035.stderr: 6000K .......... .......... .......... .......... .......... 8% 171M 1s 2019-12-09T09:27:14.992 INFO:teuthology.orchestra.run.smithi035.stderr: 6050K .......... .......... .......... .......... .......... 8% 190M 1s 2019-12-09T09:27:14.992 INFO:teuthology.orchestra.run.smithi035.stderr: 6100K .......... .......... .......... .......... .......... 8% 190M 1s 2019-12-09T09:27:14.993 INFO:teuthology.orchestra.run.smithi035.stderr: 6150K .......... .......... .......... .......... .......... 8% 149M 1s 2019-12-09T09:27:14.993 INFO:teuthology.orchestra.run.smithi035.stderr: 6200K .......... .......... .......... .......... .......... 8% 158M 1s 2019-12-09T09:27:14.993 INFO:teuthology.orchestra.run.smithi035.stderr: 6250K .......... .......... .......... .......... .......... 8% 179M 1s 2019-12-09T09:27:14.993 INFO:teuthology.orchestra.run.smithi035.stderr: 6300K .......... .......... .......... .......... .......... 8% 169M 1s 2019-12-09T09:27:14.993 INFO:teuthology.orchestra.run.smithi035.stderr: 6350K .......... .......... .......... .......... .......... 8% 132M 1s 2019-12-09T09:27:14.993 INFO:teuthology.orchestra.run.smithi035.stderr: 6400K .......... .......... .......... .......... .......... 8% 160M 1s 2019-12-09T09:27:14.993 INFO:teuthology.orchestra.run.smithi035.stderr: 6450K .......... .......... .......... .......... .......... 8% 189M 1s 2019-12-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi035.stderr: 6500K .......... .......... .......... .......... .......... 8% 159M 1s 2019-12-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi035.stderr: 6550K .......... .......... .......... .......... .......... 8% 159M 1s 2019-12-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi035.stderr: 6600K .......... .......... .......... .......... .......... 9% 149M 1s 2019-12-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi035.stderr: 6650K .......... .......... .......... .......... .......... 9% 143M 1s 2019-12-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi035.stderr: 6700K .......... .......... .......... .......... .......... 9% 152M 1s 2019-12-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi035.stderr: 6750K .......... .......... .......... .......... .......... 9% 119M 1s 2019-12-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi035.stderr: 6800K .......... .......... .......... .......... .......... 9% 154M 1s 2019-12-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi035.stderr: 6850K .......... .......... .......... .......... .......... 9% 174M 1s 2019-12-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi035.stderr: 6900K .......... .......... .......... .......... .......... 9% 145M 1s 2019-12-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi035.stderr: 6950K .......... .......... .......... .......... .......... 9% 149M 1s 2019-12-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi035.stderr: 7000K .......... .......... .......... .......... .......... 9% 132M 1s 2019-12-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi035.stderr: 7050K .......... .......... .......... .......... .......... 9% 153M 1s 2019-12-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi035.stderr: 7100K .......... .......... .......... .......... .......... 9% 150M 1s 2019-12-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi035.stderr: 7150K .......... .......... .......... .......... .......... 9% 128M 1s 2019-12-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi035.stderr: 7200K .......... .......... .......... .......... .......... 9% 146M 1s 2019-12-09T09:27:14.996 INFO:teuthology.orchestra.run.smithi035.stderr: 7250K .......... .......... .......... .......... .......... 9% 184M 1s 2019-12-09T09:27:14.996 INFO:teuthology.orchestra.run.smithi035.stderr: 7300K .......... .......... .......... .......... .......... 9% 168M 1s 2019-12-09T09:27:14.996 INFO:teuthology.orchestra.run.smithi035.stderr: 7350K .......... .......... .......... .......... .......... 10% 155M 1s 2019-12-09T09:27:14.996 INFO:teuthology.orchestra.run.smithi035.stderr: 7400K .......... .......... .......... .......... .......... 10% 185M 1s 2019-12-09T09:27:14.996 INFO:teuthology.orchestra.run.smithi035.stderr: 7450K .......... .......... .......... .......... .......... 10% 182M 1s 2019-12-09T09:27:14.996 INFO:teuthology.orchestra.run.smithi035.stderr: 7500K .......... .......... .......... .......... .......... 10% 149M 1s 2019-12-09T09:27:14.996 INFO:teuthology.orchestra.run.smithi035.stderr: 7550K .......... .......... .......... .......... .......... 10% 129M 1s 2019-12-09T09:27:14.997 INFO:teuthology.orchestra.run.smithi035.stderr: 7600K .......... .......... .......... .......... .......... 10% 173M 1s 2019-12-09T09:27:14.997 INFO:teuthology.orchestra.run.smithi035.stderr: 7650K .......... .......... .......... .......... .......... 10% 205M 1s 2019-12-09T09:27:14.997 INFO:teuthology.orchestra.run.smithi035.stderr: 7700K .......... .......... .......... .......... .......... 10% 160M 1s 2019-12-09T09:27:14.997 INFO:teuthology.orchestra.run.smithi035.stderr: 7750K .......... .......... .......... .......... .......... 10% 152M 1s 2019-12-09T09:27:14.997 INFO:teuthology.orchestra.run.smithi035.stderr: 7800K .......... .......... .......... .......... .......... 10% 180M 1s 2019-12-09T09:27:14.997 INFO:teuthology.orchestra.run.smithi035.stderr: 7850K .......... .......... .......... .......... .......... 10% 158M 1s 2019-12-09T09:27:14.997 INFO:teuthology.orchestra.run.smithi035.stderr: 7900K .......... .......... .......... .......... .......... 10% 185M 1s 2019-12-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi035.stderr: 7950K .......... .......... .......... .......... .......... 10% 139M 1s 2019-12-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi035.stderr: 8000K .......... .......... .......... .......... .......... 10% 207M 1s 2019-12-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi035.stderr: 8050K .......... .......... .......... .......... .......... 11% 166M 1s 2019-12-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi035.stderr: 8100K .......... .......... .......... .......... .......... 11% 168M 1s 2019-12-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi035.stderr: 8150K .......... .......... .......... .......... .......... 11% 169M 1s 2019-12-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi035.stderr: 8200K .......... .......... .......... .......... .......... 11% 168M 1s 2019-12-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi035.stderr: 8250K .......... .......... .......... .......... .......... 11% 160M 1s 2019-12-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi035.stderr: 8300K .......... .......... .......... .......... .......... 11% 185M 1s 2019-12-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi035.stderr: 8350K .......... .......... .......... .......... .......... 11% 159M 1s 2019-12-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi035.stderr: 8400K .......... .......... .......... .......... .......... 11% 174M 1s 2019-12-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi035.stderr: 8450K .......... .......... .......... .......... .......... 11% 169M 1s 2019-12-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi035.stderr: 8500K .......... .......... .......... .......... .......... 11% 201M 1s 2019-12-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi035.stderr: 8550K .......... .......... .......... .......... .......... 11% 170M 1s 2019-12-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi035.stderr: 8600K .......... .......... .......... .......... .......... 11% 169M 1s 2019-12-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi035.stderr: 8650K .......... .......... .......... .......... .......... 11% 168M 1s 2019-12-09T09:27:15.000 INFO:teuthology.orchestra.run.smithi035.stderr: 8700K .......... .......... .......... .......... .......... 11% 174M 1s 2019-12-09T09:27:15.000 INFO:teuthology.orchestra.run.smithi035.stderr: 8750K .......... .......... .......... .......... .......... 11% 140M 1s 2019-12-09T09:27:15.000 INFO:teuthology.orchestra.run.smithi035.stderr: 8800K .......... .......... .......... .......... .......... 12% 183M 1s 2019-12-09T09:27:15.000 INFO:teuthology.orchestra.run.smithi035.stderr: 8850K .......... .......... .......... .......... .......... 12% 166M 1s 2019-12-09T09:27:15.000 INFO:teuthology.orchestra.run.smithi035.stderr: 8900K .......... .......... .......... .......... .......... 12% 187M 1s 2019-12-09T09:27:15.000 INFO:teuthology.orchestra.run.smithi035.stderr: 8950K .......... .......... .......... .......... .......... 12% 177M 1s 2019-12-09T09:27:15.000 INFO:teuthology.orchestra.run.smithi035.stderr: 9000K .......... .......... .......... .......... .......... 12% 184M 1s 2019-12-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi035.stderr: 9050K .......... .......... .......... .......... .......... 12% 181M 1s 2019-12-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi035.stderr: 9100K .......... .......... .......... .......... .......... 12% 188M 1s 2019-12-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi035.stderr: 9150K .......... .......... .......... .......... .......... 12% 151M 1s 2019-12-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi035.stderr: 9200K .......... .......... .......... .......... .......... 12% 177M 1s 2019-12-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi035.stderr: 9250K .......... .......... .......... .......... .......... 12% 191M 1s 2019-12-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi035.stderr: 9300K .......... .......... .......... .......... .......... 12% 193M 1s 2019-12-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi035.stderr: 9350K .......... .......... .......... .......... .......... 12% 170M 1s 2019-12-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi035.stderr: 9400K .......... .......... .......... .......... .......... 12% 196M 1s 2019-12-09T09:27:15.002 INFO:teuthology.orchestra.run.smithi035.stderr: 9450K .......... .......... .......... .......... .......... 12% 192M 1s 2019-12-09T09:27:15.002 INFO:teuthology.orchestra.run.smithi035.stderr: 9500K .......... .......... .......... .......... .......... 12% 203M 1s 2019-12-09T09:27:15.002 INFO:teuthology.orchestra.run.smithi035.stderr: 9550K .......... .......... .......... .......... .......... 13% 154M 1s 2019-12-09T09:27:15.002 INFO:teuthology.orchestra.run.smithi035.stderr: 9600K .......... .......... .......... .......... .......... 13% 188M 1s 2019-12-09T09:27:15.002 INFO:teuthology.orchestra.run.smithi035.stderr: 9650K .......... .......... .......... .......... .......... 13% 189M 1s 2019-12-09T09:27:15.002 INFO:teuthology.orchestra.run.smithi035.stderr: 9700K .......... .......... .......... .......... .......... 13% 216M 1s 2019-12-09T09:27:15.002 INFO:teuthology.orchestra.run.smithi035.stderr: 9750K .......... .......... .......... .......... .......... 13% 163M 1s 2019-12-09T09:27:15.003 INFO:teuthology.orchestra.run.smithi035.stderr: 9800K .......... .......... .......... .......... .......... 13% 176M 1s 2019-12-09T09:27:15.003 INFO:teuthology.orchestra.run.smithi035.stderr: 9850K .......... .......... .......... .......... .......... 13% 176M 1s 2019-12-09T09:27:15.003 INFO:teuthology.orchestra.run.smithi035.stderr: 9900K .......... .......... .......... .......... .......... 13% 168M 1s 2019-12-09T09:27:15.003 INFO:teuthology.orchestra.run.smithi035.stderr: 9950K .......... .......... .......... .......... .......... 13% 142M 1s 2019-12-09T09:27:15.003 INFO:teuthology.orchestra.run.smithi035.stderr: 10000K .......... .......... .......... .......... .......... 13% 207M 1s 2019-12-09T09:27:15.003 INFO:teuthology.orchestra.run.smithi035.stderr: 10050K .......... .......... .......... .......... .......... 13% 183M 1s 2019-12-09T09:27:15.003 INFO:teuthology.orchestra.run.smithi035.stderr: 10100K .......... .......... .......... .......... .......... 13% 178M 1s 2019-12-09T09:27:15.003 INFO:teuthology.orchestra.run.smithi035.stderr: 10150K .......... .......... .......... .......... .......... 13% 176M 1s 2019-12-09T09:27:15.004 INFO:teuthology.orchestra.run.smithi035.stderr: 10200K .......... .......... .......... .......... .......... 13% 150M 1s 2019-12-09T09:27:15.004 INFO:teuthology.orchestra.run.smithi035.stderr: 10250K .......... .......... .......... .......... .......... 14% 164M 1s 2019-12-09T09:27:15.004 INFO:teuthology.orchestra.run.smithi035.stderr: 10300K .......... .......... .......... .......... .......... 14% 200M 1s 2019-12-09T09:27:15.004 INFO:teuthology.orchestra.run.smithi035.stderr: 10350K .......... .......... .......... .......... .......... 14% 133M 1s 2019-12-09T09:27:15.004 INFO:teuthology.orchestra.run.smithi035.stderr: 10400K .......... .......... .......... .......... .......... 14% 189M 1s 2019-12-09T09:27:15.004 INFO:teuthology.orchestra.run.smithi035.stderr: 10450K .......... .......... .......... .......... .......... 14% 167M 1s 2019-12-09T09:27:15.004 INFO:teuthology.orchestra.run.smithi035.stderr: 10500K .......... .......... .......... .......... .......... 14% 174M 1s 2019-12-09T09:27:15.005 INFO:teuthology.orchestra.run.smithi035.stderr: 10550K .......... .......... .......... .......... .......... 14% 166M 1s 2019-12-09T09:27:15.005 INFO:teuthology.orchestra.run.smithi035.stderr: 10600K .......... .......... .......... .......... .......... 14% 204M 1s 2019-12-09T09:27:15.005 INFO:teuthology.orchestra.run.smithi035.stderr: 10650K .......... .......... .......... .......... .......... 14% 165M 1s 2019-12-09T09:27:15.005 INFO:teuthology.orchestra.run.smithi035.stderr: 10700K .......... .......... .......... .......... .......... 14% 159M 1s 2019-12-09T09:27:15.005 INFO:teuthology.orchestra.run.smithi035.stderr: 10750K .......... .......... .......... .......... .......... 14% 141M 1s 2019-12-09T09:27:15.005 INFO:teuthology.orchestra.run.smithi035.stderr: 10800K .......... .......... .......... .......... .......... 14% 172M 1s 2019-12-09T09:27:15.005 INFO:teuthology.orchestra.run.smithi035.stderr: 10850K .......... .......... .......... .......... .......... 14% 169M 1s 2019-12-09T09:27:15.006 INFO:teuthology.orchestra.run.smithi035.stderr: 10900K .......... .......... .......... .......... .......... 14% 193M 1s 2019-12-09T09:27:15.006 INFO:teuthology.orchestra.run.smithi035.stderr: 10950K .......... .......... .......... .......... .......... 14% 186M 1s 2019-12-09T09:27:15.006 INFO:teuthology.orchestra.run.smithi035.stderr: 11000K .......... .......... .......... .......... .......... 15% 209M 1s 2019-12-09T09:27:15.006 INFO:teuthology.orchestra.run.smithi035.stderr: 11050K .......... .......... .......... .......... .......... 15% 203M 1s 2019-12-09T09:27:15.006 INFO:teuthology.orchestra.run.smithi035.stderr: 11100K .......... .......... .......... .......... .......... 15% 206M 1s 2019-12-09T09:27:15.006 INFO:teuthology.orchestra.run.smithi035.stderr: 11150K .......... .......... .......... .......... .......... 15% 178M 1s 2019-12-09T09:27:15.006 INFO:teuthology.orchestra.run.smithi035.stderr: 11200K .......... .......... .......... .......... .......... 15% 211M 1s 2019-12-09T09:27:15.006 INFO:teuthology.orchestra.run.smithi035.stderr: 11250K .......... .......... .......... .......... .......... 15% 205M 1s 2019-12-09T09:27:15.007 INFO:teuthology.orchestra.run.smithi035.stderr: 11300K .......... .......... .......... .......... .......... 15% 211M 1s 2019-12-09T09:27:15.007 INFO:teuthology.orchestra.run.smithi035.stderr: 11350K .......... .......... .......... .......... .......... 15% 196M 1s 2019-12-09T09:27:15.007 INFO:teuthology.orchestra.run.smithi035.stderr: 11400K .......... .......... .......... .......... .......... 15% 228M 1s 2019-12-09T09:27:15.007 INFO:teuthology.orchestra.run.smithi035.stderr: 11450K .......... .......... .......... .......... .......... 15% 218M 1s 2019-12-09T09:27:15.007 INFO:teuthology.orchestra.run.smithi035.stderr: 11500K .......... .......... .......... .......... .......... 15% 203M 1s 2019-12-09T09:27:15.007 INFO:teuthology.orchestra.run.smithi035.stderr: 11550K .......... .......... .......... .......... .......... 15% 188M 1s 2019-12-09T09:27:15.007 INFO:teuthology.orchestra.run.smithi035.stderr: 11600K .......... .......... .......... .......... .......... 15% 187M 1s 2019-12-09T09:27:15.008 INFO:teuthology.orchestra.run.smithi035.stderr: 11650K .......... .......... .......... .......... .......... 15% 216M 1s 2019-12-09T09:27:15.008 INFO:teuthology.orchestra.run.smithi035.stderr: 11700K .......... .......... .......... .......... .......... 15% 192M 1s 2019-12-09T09:27:15.008 INFO:teuthology.orchestra.run.smithi035.stderr: 11750K .......... .......... .......... .......... .......... 16% 198M 1s 2019-12-09T09:27:15.008 INFO:teuthology.orchestra.run.smithi035.stderr: 11800K .......... .......... .......... .......... .......... 16% 177M 1s 2019-12-09T09:27:15.008 INFO:teuthology.orchestra.run.smithi035.stderr: 11850K .......... .......... .......... .......... .......... 16% 196M 1s 2019-12-09T09:27:15.008 INFO:teuthology.orchestra.run.smithi035.stderr: 11900K .......... .......... .......... .......... .......... 16% 202M 1s 2019-12-09T09:27:15.008 INFO:teuthology.orchestra.run.smithi035.stderr: 11950K .......... .......... .......... .......... .......... 16% 143M 1s 2019-12-09T09:27:15.009 INFO:teuthology.orchestra.run.smithi035.stderr: 12000K .......... .......... .......... .......... .......... 16% 230M 1s 2019-12-09T09:27:15.009 INFO:teuthology.orchestra.run.smithi035.stderr: 12050K .......... .......... .......... .......... .......... 16% 211M 1s 2019-12-09T09:27:15.009 INFO:teuthology.orchestra.run.smithi035.stderr: 12100K .......... .......... .......... .......... .......... 16% 187M 1s 2019-12-09T09:27:15.009 INFO:teuthology.orchestra.run.smithi035.stderr: 12150K .......... .......... .......... .......... .......... 16% 141M 1s 2019-12-09T09:27:15.009 INFO:teuthology.orchestra.run.smithi035.stderr: 12200K .......... .......... .......... .......... .......... 16% 187M 1s 2019-12-09T09:27:15.009 INFO:teuthology.orchestra.run.smithi035.stderr: 12250K .......... .......... .......... .......... .......... 16% 195M 1s 2019-12-09T09:27:15.009 INFO:teuthology.orchestra.run.smithi035.stderr: 12300K .......... .......... .......... .......... .......... 16% 180M 1s 2019-12-09T09:27:15.009 INFO:teuthology.orchestra.run.smithi035.stderr: 12350K .......... .......... .......... .......... .......... 16% 152M 1s 2019-12-09T09:27:15.010 INFO:teuthology.orchestra.run.smithi035.stderr: 12400K .......... .......... .......... .......... .......... 16% 201M 1s 2019-12-09T09:27:15.010 INFO:teuthology.orchestra.run.smithi035.stderr: 12450K .......... .......... .......... .......... .......... 16% 171M 1s 2019-12-09T09:27:15.010 INFO:teuthology.orchestra.run.smithi035.stderr: 12500K .......... .......... .......... .......... .......... 17% 200M 1s 2019-12-09T09:27:15.010 INFO:teuthology.orchestra.run.smithi035.stderr: 12550K .......... .......... .......... .......... .......... 17% 155M 1s 2019-12-09T09:27:15.010 INFO:teuthology.orchestra.run.smithi035.stderr: 12600K .......... .......... .......... .......... .......... 17% 151M 0s 2019-12-09T09:27:15.010 INFO:teuthology.orchestra.run.smithi035.stderr: 12650K .......... .......... .......... .......... .......... 17% 169M 0s 2019-12-09T09:27:15.011 INFO:teuthology.orchestra.run.smithi035.stderr: 12700K .......... .......... .......... .......... .......... 17% 180M 0s 2019-12-09T09:27:15.011 INFO:teuthology.orchestra.run.smithi035.stderr: 12750K .......... .......... .......... .......... .......... 17% 159M 0s 2019-12-09T09:27:15.011 INFO:teuthology.orchestra.run.smithi035.stderr: 12800K .......... .......... .......... .......... .......... 17% 215M 0s 2019-12-09T09:27:15.011 INFO:teuthology.orchestra.run.smithi035.stderr: 12850K .......... .......... .......... .......... .......... 17% 172M 0s 2019-12-09T09:27:15.011 INFO:teuthology.orchestra.run.smithi035.stderr: 12900K .......... .......... .......... .......... .......... 17% 196M 0s 2019-12-09T09:27:15.011 INFO:teuthology.orchestra.run.smithi035.stderr: 12950K .......... .......... .......... .......... .......... 17% 172M 0s 2019-12-09T09:27:15.012 INFO:teuthology.orchestra.run.smithi035.stderr: 13000K .......... .......... .......... .......... .......... 17% 202M 0s 2019-12-09T09:27:15.012 INFO:teuthology.orchestra.run.smithi035.stderr: 13050K .......... .......... .......... .......... .......... 17% 229M 0s 2019-12-09T09:27:15.012 INFO:teuthology.orchestra.run.smithi035.stderr: 13100K .......... .......... .......... .......... .......... 17% 209M 0s 2019-12-09T09:27:15.012 INFO:teuthology.orchestra.run.smithi035.stderr: 13150K .......... .......... .......... .......... .......... 17% 198M 0s 2019-12-09T09:27:15.012 INFO:teuthology.orchestra.run.smithi035.stderr: 13200K .......... .......... .......... .......... .......... 18% 206M 0s 2019-12-09T09:27:15.012 INFO:teuthology.orchestra.run.smithi035.stderr: 13250K .......... .......... .......... .......... .......... 18% 214M 0s 2019-12-09T09:27:15.012 INFO:teuthology.orchestra.run.smithi035.stderr: 13300K .......... .......... .......... .......... .......... 18% 229M 0s 2019-12-09T09:27:15.013 INFO:teuthology.orchestra.run.smithi035.stderr: 13350K .......... .......... .......... .......... .......... 18% 175M 0s 2019-12-09T09:27:15.013 INFO:teuthology.orchestra.run.smithi035.stderr: 13400K .......... .......... .......... .......... .......... 18% 195M 0s 2019-12-09T09:27:15.013 INFO:teuthology.orchestra.run.smithi035.stderr: 13450K .......... .......... .......... .......... .......... 18% 237M 0s 2019-12-09T09:27:15.013 INFO:teuthology.orchestra.run.smithi035.stderr: 13500K .......... .......... .......... .......... .......... 18% 236M 0s 2019-12-09T09:27:15.013 INFO:teuthology.orchestra.run.smithi035.stderr: 13550K .......... .......... .......... .......... .......... 18% 182M 0s 2019-12-09T09:27:15.013 INFO:teuthology.orchestra.run.smithi035.stderr: 13600K .......... .......... .......... .......... .......... 18% 174M 0s 2019-12-09T09:27:15.013 INFO:teuthology.orchestra.run.smithi035.stderr: 13650K .......... .......... .......... .......... .......... 18% 189M 0s 2019-12-09T09:27:15.013 INFO:teuthology.orchestra.run.smithi035.stderr: 13700K .......... .......... .......... .......... .......... 18% 206M 0s 2019-12-09T09:27:15.014 INFO:teuthology.orchestra.run.smithi035.stderr: 13750K .......... .......... .......... .......... .......... 18% 168M 0s 2019-12-09T09:27:15.014 INFO:teuthology.orchestra.run.smithi035.stderr: 13800K .......... .......... .......... .......... .......... 18% 199M 0s 2019-12-09T09:27:15.014 INFO:teuthology.orchestra.run.smithi035.stderr: 13850K .......... .......... .......... .......... .......... 18% 241M 0s 2019-12-09T09:27:15.014 INFO:teuthology.orchestra.run.smithi035.stderr: 13900K .......... .......... .......... .......... .......... 18% 204M 0s 2019-12-09T09:27:15.014 INFO:teuthology.orchestra.run.smithi035.stderr: 13950K .......... .......... .......... .......... .......... 19% 168M 0s 2019-12-09T09:27:15.014 INFO:teuthology.orchestra.run.smithi035.stderr: 14000K .......... .......... .......... .......... .......... 19% 177M 0s 2019-12-09T09:27:15.014 INFO:teuthology.orchestra.run.smithi035.stderr: 14050K .......... .......... .......... .......... .......... 19% 197M 0s 2019-12-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi035.stderr: 14100K .......... .......... .......... .......... .......... 19% 195M 0s 2019-12-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi035.stderr: 14150K .......... .......... .......... .......... .......... 19% 171M 0s 2019-12-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi035.stderr: 14200K .......... .......... .......... .......... .......... 19% 203M 0s 2019-12-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi035.stderr: 14250K .......... .......... .......... .......... .......... 19% 190M 0s 2019-12-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi035.stderr: 14300K .......... .......... .......... .......... .......... 19% 224M 0s 2019-12-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi035.stderr: 14350K .......... .......... .......... .......... .......... 19% 160M 0s 2019-12-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi035.stderr: 14400K .......... .......... .......... .......... .......... 19% 192M 0s 2019-12-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi035.stderr: 14450K .......... .......... .......... .......... .......... 19% 219M 0s 2019-12-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi035.stderr: 14500K .......... .......... .......... .......... .......... 19% 208M 0s 2019-12-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi035.stderr: 14550K .......... .......... .......... .......... .......... 19% 206M 0s 2019-12-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi035.stderr: 14600K .......... .......... .......... .......... .......... 19% 213M 0s 2019-12-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi035.stderr: 14650K .......... .......... .......... .......... .......... 19% 239M 0s 2019-12-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi035.stderr: 14700K .......... .......... .......... .......... .......... 20% 235M 0s 2019-12-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi035.stderr: 14750K .......... .......... .......... .......... .......... 20% 199M 0s 2019-12-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi035.stderr: 14800K .......... .......... .......... .......... .......... 20% 219M 0s 2019-12-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi035.stderr: 14850K .......... .......... .......... .......... .......... 20% 231M 0s 2019-12-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi035.stderr: 14900K .......... .......... .......... .......... .......... 20% 218M 0s 2019-12-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi035.stderr: 14950K .......... .......... .......... .......... .......... 20% 201M 0s 2019-12-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi035.stderr: 15000K .......... .......... .......... .......... .......... 20% 240M 0s 2019-12-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi035.stderr: 15050K .......... .......... .......... .......... .......... 20% 229M 0s 2019-12-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi035.stderr: 15100K .......... .......... .......... .......... .......... 20% 242M 0s 2019-12-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi035.stderr: 15150K .......... .......... .......... .......... .......... 20% 168M 0s 2019-12-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi035.stderr: 15200K .......... .......... .......... .......... .......... 20% 210M 0s 2019-12-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi035.stderr: 15250K .......... .......... .......... .......... .......... 20% 212M 0s 2019-12-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi035.stderr: 15300K .......... .......... .......... .......... .......... 20% 219M 0s 2019-12-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi035.stderr: 15350K .......... .......... .......... .......... .......... 20% 192M 0s 2019-12-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi035.stderr: 15400K .......... .......... .......... .......... .......... 21% 221M 0s 2019-12-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi035.stderr: 15450K .......... .......... .......... .......... .......... 21% 249M 0s 2019-12-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi035.stderr: 15500K .......... .......... .......... .......... .......... 21% 212M 0s 2019-12-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi035.stderr: 15550K .......... .......... .......... .......... .......... 21% 190M 0s 2019-12-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi035.stderr: 15600K .......... .......... .......... .......... .......... 21% 219M 0s 2019-12-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi035.stderr: 15650K .......... .......... .......... .......... .......... 21% 218M 0s 2019-12-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi035.stderr: 15700K .......... .......... .......... .......... .......... 21% 193M 0s 2019-12-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi035.stderr: 15750K .......... .......... .......... .......... .......... 21% 179M 0s 2019-12-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi035.stderr: 15800K .......... .......... .......... .......... .......... 21% 218M 0s 2019-12-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi035.stderr: 15850K .......... .......... .......... .......... .......... 21% 198M 0s 2019-12-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi035.stderr: 15900K .......... .......... .......... .......... .......... 21% 221M 0s 2019-12-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi035.stderr: 15950K .......... .......... .......... .......... .......... 21% 159M 0s 2019-12-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi035.stderr: 16000K .......... .......... .......... .......... .......... 21% 227M 0s 2019-12-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi035.stderr: 16050K .......... .......... .......... .......... .......... 21% 183M 0s 2019-12-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi035.stderr: 16100K .......... .......... .......... .......... .......... 21% 205M 0s 2019-12-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi035.stderr: 16150K .......... .......... .......... .......... .......... 22% 213M 0s 2019-12-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi035.stderr: 16200K .......... .......... .......... .......... .......... 22% 195M 0s 2019-12-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi035.stderr: 16250K .......... .......... .......... .......... .......... 22% 216M 0s 2019-12-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi035.stderr: 16300K .......... .......... .......... .......... .......... 22% 220M 0s 2019-12-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi035.stderr: 16350K .......... .......... .......... .......... .......... 22% 153M 0s 2019-12-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi035.stderr: 16400K .......... .......... .......... .......... .......... 22% 242M 0s 2019-12-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi035.stderr: 16450K .......... .......... .......... .......... .......... 22% 219M 0s 2019-12-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi035.stderr: 16500K .......... .......... .......... .......... .......... 22% 203M 0s 2019-12-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi035.stderr: 16550K .......... .......... .......... .......... .......... 22% 206M 0s 2019-12-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi035.stderr: 16600K .......... .......... .......... .......... .......... 22% 229M 0s 2019-12-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi035.stderr: 16650K .......... .......... .......... .......... .......... 22% 238M 0s 2019-12-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi035.stderr: 16700K .......... .......... .......... .......... .......... 22% 225M 0s 2019-12-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi035.stderr: 16750K .......... .......... .......... .......... .......... 22% 163M 0s 2019-12-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi035.stderr: 16800K .......... .......... .......... .......... .......... 22% 205M 0s 2019-12-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi035.stderr: 16850K .......... .......... .......... .......... .......... 22% 244M 0s 2019-12-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi035.stderr: 16900K .......... .......... .......... .......... .......... 23% 230M 0s 2019-12-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi035.stderr: 16950K .......... .......... .......... .......... .......... 23% 207M 0s 2019-12-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi035.stderr: 17000K .......... .......... .......... .......... .......... 23% 243M 0s 2019-12-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi035.stderr: 17050K .......... .......... .......... .......... .......... 23% 256M 0s 2019-12-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi035.stderr: 17100K .......... .......... .......... .......... .......... 23% 248M 0s 2019-12-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi035.stderr: 17150K .......... .......... .......... .......... .......... 23% 203M 0s 2019-12-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi035.stderr: 17200K .......... .......... .......... .......... .......... 23% 249M 0s 2019-12-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi035.stderr: 17250K .......... .......... .......... .......... .......... 23% 248M 0s 2019-12-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi035.stderr: 17300K .......... .......... .......... .......... .......... 23% 228M 0s 2019-12-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi035.stderr: 17350K .......... .......... .......... .......... .......... 23% 212M 0s 2019-12-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi035.stderr: 17400K .......... .......... .......... .......... .......... 23% 261M 0s 2019-12-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi035.stderr: 17450K .......... .......... .......... .......... .......... 23% 227M 0s 2019-12-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi035.stderr: 17500K .......... .......... .......... .......... .......... 23% 252M 0s 2019-12-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi035.stderr: 17550K .......... .......... .......... .......... .......... 23% 175M 0s 2019-12-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi035.stderr: 17600K .......... .......... .......... .......... .......... 23% 216M 0s 2019-12-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi035.stderr: 17650K .......... .......... .......... .......... .......... 24% 217M 0s 2019-12-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi035.stderr: 17700K .......... .......... .......... .......... .......... 24% 198M 0s 2019-12-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi035.stderr: 17750K .......... .......... .......... .......... .......... 24% 141M 0s 2019-12-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi035.stderr: 17800K .......... .......... .......... .......... .......... 24% 155M 0s 2019-12-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi035.stderr: 17850K .......... .......... .......... .......... .......... 24% 163M 0s 2019-12-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi035.stderr: 17900K .......... .......... .......... .......... .......... 24% 155M 0s 2019-12-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi035.stderr: 17950K .......... .......... .......... .......... .......... 24% 129M 0s 2019-12-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi035.stderr: 18000K .......... .......... .......... .......... .......... 24% 167M 0s 2019-12-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi035.stderr: 18050K .......... .......... .......... .......... .......... 24% 152M 0s 2019-12-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi035.stderr: 18100K .......... .......... .......... .......... .......... 24% 167M 0s 2019-12-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi035.stderr: 18150K .......... .......... .......... .......... .......... 24% 138M 0s 2019-12-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi035.stderr: 18200K .......... .......... .......... .......... .......... 24% 172M 0s 2019-12-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi035.stderr: 18250K .......... .......... .......... .......... .......... 24% 151M 0s 2019-12-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi035.stderr: 18300K .......... .......... .......... .......... .......... 24% 159M 0s 2019-12-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi035.stderr: 18350K .......... .......... .......... .......... .......... 25% 125M 0s 2019-12-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi035.stderr: 18400K .......... .......... .......... .......... .......... 25% 162M 0s 2019-12-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi035.stderr: 18450K .......... .......... .......... .......... .......... 25% 165M 0s 2019-12-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi035.stderr: 18500K .......... .......... .......... .......... .......... 25% 167M 0s 2019-12-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi035.stderr: 18550K .......... .......... .......... .......... .......... 25% 152M 0s 2019-12-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi035.stderr: 18600K .......... .......... .......... .......... .......... 25% 156M 0s 2019-12-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi035.stderr: 18650K .......... .......... .......... .......... .......... 25% 166M 0s 2019-12-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi035.stderr: 18700K .......... .......... .......... .......... .......... 25% 154M 0s 2019-12-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi035.stderr: 18750K .......... .......... .......... .......... .......... 25% 137M 0s 2019-12-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi035.stderr: 18800K .......... .......... .......... .......... .......... 25% 163M 0s 2019-12-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi035.stderr: 18850K .......... .......... .......... .......... .......... 25% 164M 0s 2019-12-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi035.stderr: 18900K .......... .......... .......... .......... .......... 25% 175M 0s 2019-12-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi035.stderr: 18950K .......... .......... .......... .......... .......... 25% 148M 0s 2019-12-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi035.stderr: 19000K .......... .......... .......... .......... .......... 25% 164M 0s 2019-12-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi035.stderr: 19050K .......... .......... .......... .......... .......... 25% 178M 0s 2019-12-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi035.stderr: 19100K .......... .......... .......... .......... .......... 26% 169M 0s 2019-12-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi035.stderr: 19150K .......... .......... .......... .......... .......... 26% 143M 0s 2019-12-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi035.stderr: 19200K .......... .......... .......... .......... .......... 26% 173M 0s 2019-12-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi035.stderr: 19250K .......... .......... .......... .......... .......... 26% 175M 0s 2019-12-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi035.stderr: 19300K .......... .......... .......... .......... .......... 26% 170M 0s 2019-12-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi035.stderr: 19350K .......... .......... .......... .......... .......... 26% 151M 0s 2019-12-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi035.stderr: 19400K .......... .......... .......... .......... .......... 26% 170M 0s 2019-12-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi035.stderr: 19450K .......... .......... .......... .......... .......... 26% 167M 0s 2019-12-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi035.stderr: 19500K .......... .......... .......... .......... .......... 26% 175M 0s 2019-12-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi035.stderr: 19550K .......... .......... .......... .......... .......... 26% 148M 0s 2019-12-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi035.stderr: 19600K .......... .......... .......... .......... .......... 26% 183M 0s 2019-12-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi035.stderr: 19650K .......... .......... .......... .......... .......... 26% 175M 0s 2019-12-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi035.stderr: 19700K .......... .......... .......... .......... .......... 26% 175M 0s 2019-12-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi035.stderr: 19750K .......... .......... .......... .......... .......... 26% 159M 0s 2019-12-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi035.stderr: 19800K .......... .......... .......... .......... .......... 26% 161M 0s 2019-12-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi035.stderr: 19850K .......... .......... .......... .......... .......... 27% 176M 0s 2019-12-09T09:27:15.031 INFO:teuthology.orchestra.run.smithi035.stderr: 19900K .......... .......... .......... .......... .......... 27% 167M 0s 2019-12-09T09:27:15.031 INFO:teuthology.orchestra.run.smithi035.stderr: 19950K .......... .......... .......... .......... .......... 27% 133M 0s 2019-12-09T09:27:15.031 INFO:teuthology.orchestra.run.smithi035.stderr: 20000K .......... .......... .......... .......... .......... 27% 218M 0s 2019-12-09T09:27:15.031 INFO:teuthology.orchestra.run.smithi035.stderr: 20050K .......... .......... .......... .......... .......... 27% 251M 0s 2019-12-09T09:27:15.031 INFO:teuthology.orchestra.run.smithi035.stderr: 20100K .......... .......... .......... .......... .......... 27% 211M 0s 2019-12-09T09:27:15.031 INFO:teuthology.orchestra.run.smithi035.stderr: 20150K .......... .......... .......... .......... .......... 27% 197M 0s 2019-12-09T09:27:15.031 INFO:teuthology.orchestra.run.smithi035.stderr: 20200K .......... .......... .......... .......... .......... 27% 253M 0s 2019-12-09T09:27:15.031 INFO:teuthology.orchestra.run.smithi035.stderr: 20250K .......... .......... .......... .......... .......... 27% 221M 0s 2019-12-09T09:27:15.032 INFO:teuthology.orchestra.run.smithi035.stderr: 20300K .......... .......... .......... .......... .......... 27% 222M 0s 2019-12-09T09:27:15.032 INFO:teuthology.orchestra.run.smithi035.stderr: 20350K .......... .......... .......... .......... .......... 27% 193M 0s 2019-12-09T09:27:15.032 INFO:teuthology.orchestra.run.smithi035.stderr: 20400K .......... .......... .......... .......... .......... 27% 239M 0s 2019-12-09T09:27:15.032 INFO:teuthology.orchestra.run.smithi035.stderr: 20450K .......... .......... .......... .......... .......... 27% 250M 0s 2019-12-09T09:27:15.032 INFO:teuthology.orchestra.run.smithi035.stderr: 20500K .......... .......... .......... .......... .......... 27% 241M 0s 2019-12-09T09:27:15.032 INFO:teuthology.orchestra.run.smithi035.stderr: 20550K .......... .......... .......... .......... .......... 28% 212M 0s 2019-12-09T09:27:15.032 INFO:teuthology.orchestra.run.smithi035.stderr: 20600K .......... .......... .......... .......... .......... 28% 230M 0s 2019-12-09T09:27:15.033 INFO:teuthology.orchestra.run.smithi035.stderr: 20650K .......... .......... .......... .......... .......... 28% 240M 0s 2019-12-09T09:27:15.033 INFO:teuthology.orchestra.run.smithi035.stderr: 20700K .......... .......... .......... .......... .......... 28% 265M 0s 2019-12-09T09:27:15.033 INFO:teuthology.orchestra.run.smithi035.stderr: 20750K .......... .......... .......... .......... .......... 28% 197M 0s 2019-12-09T09:27:15.033 INFO:teuthology.orchestra.run.smithi035.stderr: 20800K .......... .......... .......... .......... .......... 28% 234M 0s 2019-12-09T09:27:15.033 INFO:teuthology.orchestra.run.smithi035.stderr: 20850K .......... .......... .......... .......... .......... 28% 245M 0s 2019-12-09T09:27:15.033 INFO:teuthology.orchestra.run.smithi035.stderr: 20900K .......... .......... .......... .......... .......... 28% 229M 0s 2019-12-09T09:27:15.033 INFO:teuthology.orchestra.run.smithi035.stderr: 20950K .......... .......... .......... .......... .......... 28% 216M 0s 2019-12-09T09:27:15.034 INFO:teuthology.orchestra.run.smithi035.stderr: 21000K .......... .......... .......... .......... .......... 28% 251M 0s 2019-12-09T09:27:15.034 INFO:teuthology.orchestra.run.smithi035.stderr: 21050K .......... .......... .......... .......... .......... 28% 224M 0s 2019-12-09T09:27:15.034 INFO:teuthology.orchestra.run.smithi035.stderr: 21100K .......... .......... .......... .......... .......... 28% 220M 0s 2019-12-09T09:27:15.034 INFO:teuthology.orchestra.run.smithi035.stderr: 21150K .......... .......... .......... .......... .......... 28% 181M 0s 2019-12-09T09:27:15.034 INFO:teuthology.orchestra.run.smithi035.stderr: 21200K .......... .......... .......... .......... .......... 28% 246M 0s 2019-12-09T09:27:15.034 INFO:teuthology.orchestra.run.smithi035.stderr: 21250K .......... .......... .......... .......... .......... 28% 235M 0s 2019-12-09T09:27:15.034 INFO:teuthology.orchestra.run.smithi035.stderr: 21300K .......... .......... .......... .......... .......... 29% 254M 0s 2019-12-09T09:27:15.034 INFO:teuthology.orchestra.run.smithi035.stderr: 21350K .......... .......... .......... .......... .......... 29% 202M 0s 2019-12-09T09:27:15.035 INFO:teuthology.orchestra.run.smithi035.stderr: 21400K .......... .......... .......... .......... .......... 29% 234M 0s 2019-12-09T09:27:15.035 INFO:teuthology.orchestra.run.smithi035.stderr: 21450K .......... .......... .......... .......... .......... 29% 224M 0s 2019-12-09T09:27:15.035 INFO:teuthology.orchestra.run.smithi035.stderr: 21500K .......... .......... .......... .......... .......... 29% 157M 0s 2019-12-09T09:27:15.035 INFO:teuthology.orchestra.run.smithi035.stderr: 21550K .......... .......... .......... .......... .......... 29% 150M 0s 2019-12-09T09:27:15.035 INFO:teuthology.orchestra.run.smithi035.stderr: 21600K .......... .......... .......... .......... .......... 29% 160M 0s 2019-12-09T09:27:15.035 INFO:teuthology.orchestra.run.smithi035.stderr: 21650K .......... .......... .......... .......... .......... 29% 155M 0s 2019-12-09T09:27:15.035 INFO:teuthology.orchestra.run.smithi035.stderr: 21700K .......... .......... .......... .......... .......... 29% 154M 0s 2019-12-09T09:27:15.036 INFO:teuthology.orchestra.run.smithi035.stderr: 21750K .......... .......... .......... .......... .......... 29% 141M 0s 2019-12-09T09:27:15.036 INFO:teuthology.orchestra.run.smithi035.stderr: 21800K .......... .......... .......... .......... .......... 29% 160M 0s 2019-12-09T09:27:15.036 INFO:teuthology.orchestra.run.smithi035.stderr: 21850K .......... .......... .......... .......... .......... 29% 148M 0s 2019-12-09T09:27:15.036 INFO:teuthology.orchestra.run.smithi035.stderr: 21900K .......... .......... .......... .......... .......... 29% 153M 0s 2019-12-09T09:27:15.036 INFO:teuthology.orchestra.run.smithi035.stderr: 21950K .......... .......... .......... .......... .......... 29% 133M 0s 2019-12-09T09:27:15.036 INFO:teuthology.orchestra.run.smithi035.stderr: 22000K .......... .......... .......... .......... .......... 29% 149M 0s 2019-12-09T09:27:15.036 INFO:teuthology.orchestra.run.smithi035.stderr: 22050K .......... .......... .......... .......... .......... 30% 152M 0s 2019-12-09T09:27:15.037 INFO:teuthology.orchestra.run.smithi035.stderr: 22100K .......... .......... .......... .......... .......... 30% 179M 0s 2019-12-09T09:27:15.037 INFO:teuthology.orchestra.run.smithi035.stderr: 22150K .......... .......... .......... .......... .......... 30% 148M 0s 2019-12-09T09:27:15.037 INFO:teuthology.orchestra.run.smithi035.stderr: 22200K .......... .......... .......... .......... .......... 30% 240M 0s 2019-12-09T09:27:15.037 INFO:teuthology.orchestra.run.smithi035.stderr: 22250K .......... .......... .......... .......... .......... 30% 220M 0s 2019-12-09T09:27:15.037 INFO:teuthology.orchestra.run.smithi035.stderr: 22300K .......... .......... .......... .......... .......... 30% 150M 0s 2019-12-09T09:27:15.037 INFO:teuthology.orchestra.run.smithi035.stderr: 22350K .......... .......... .......... .......... .......... 30% 135M 0s 2019-12-09T09:27:15.037 INFO:teuthology.orchestra.run.smithi035.stderr: 22400K .......... .......... .......... .......... .......... 30% 200M 0s 2019-12-09T09:27:15.037 INFO:teuthology.orchestra.run.smithi035.stderr: 22450K .......... .......... .......... .......... .......... 30% 211M 0s 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi035.stderr: 22500K .......... .......... .......... .......... .......... 30% 181M 0s 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi035.stderr: 22550K .......... .......... .......... .......... .......... 30% 175M 0s 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi035.stderr: 22600K .......... .......... .......... .......... .......... 30% 188M 0s 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi035.stderr: 22650K .......... .......... .......... .......... .......... 30% 161M 0s 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi035.stderr: 22700K .......... .......... .......... .......... .......... 30% 199M 0s 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi035.stderr: 22750K .......... .......... .......... .......... .......... 30% 222M 0s 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi035.stderr: 22800K .......... .......... .......... .......... .......... 31% 241M 0s 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi035.stderr: 22850K .......... .......... .......... .......... .......... 31% 261M 0s 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi035.stderr: 22900K .......... .......... .......... .......... .......... 31% 235M 0s 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi035.stderr: 22950K .......... .......... .......... .......... .......... 31% 190M 0s 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi035.stderr: 23000K .......... .......... .......... .......... .......... 31% 151M 0s 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi035.stderr: 23050K .......... .......... .......... .......... .......... 31% 163M 0s 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi035.stderr: 23100K .......... .......... .......... .......... .......... 31% 195M 0s 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi035.stderr: 23150K .......... .......... .......... .......... .......... 31% 219M 0s 2019-12-09T09:27:15.040 INFO:teuthology.orchestra.run.smithi035.stderr: 23200K .......... .......... .......... .......... .......... 31% 228M 0s 2019-12-09T09:27:15.040 INFO:teuthology.orchestra.run.smithi035.stderr: 23250K .......... .......... .......... .......... .......... 31% 241M 0s 2019-12-09T09:27:15.040 INFO:teuthology.orchestra.run.smithi035.stderr: 23300K .......... .......... .......... .......... .......... 31% 264M 0s 2019-12-09T09:27:15.040 INFO:teuthology.orchestra.run.smithi035.stderr: 23350K .......... .......... .......... .......... .......... 31% 205M 0s 2019-12-09T09:27:15.040 INFO:teuthology.orchestra.run.smithi035.stderr: 23400K .......... .......... .......... .......... .......... 31% 277M 0s 2019-12-09T09:27:15.040 INFO:teuthology.orchestra.run.smithi035.stderr: 23450K .......... .......... .......... .......... .......... 31% 252M 0s 2019-12-09T09:27:15.040 INFO:teuthology.orchestra.run.smithi035.stderr: 23500K .......... .......... .......... .......... .......... 32% 263M 0s 2019-12-09T09:27:15.041 INFO:teuthology.orchestra.run.smithi035.stderr: 23550K .......... .......... .......... .......... .......... 32% 238M 0s 2019-12-09T09:27:15.041 INFO:teuthology.orchestra.run.smithi035.stderr: 23600K .......... .......... .......... .......... .......... 32% 201M 0s 2019-12-09T09:27:15.041 INFO:teuthology.orchestra.run.smithi035.stderr: 23650K .......... .......... .......... .......... .......... 32% 176M 0s 2019-12-09T09:27:15.041 INFO:teuthology.orchestra.run.smithi035.stderr: 23700K .......... .......... .......... .......... .......... 32% 180M 0s 2019-12-09T09:27:15.041 INFO:teuthology.orchestra.run.smithi035.stderr: 23750K .......... .......... .......... .......... .......... 32% 171M 0s 2019-12-09T09:27:15.041 INFO:teuthology.orchestra.run.smithi035.stderr: 23800K .......... .......... .......... .......... .......... 32% 192M 0s 2019-12-09T09:27:15.041 INFO:teuthology.orchestra.run.smithi035.stderr: 23850K .......... .......... .......... .......... .......... 32% 184M 0s 2019-12-09T09:27:15.041 INFO:teuthology.orchestra.run.smithi035.stderr: 23900K .......... .......... .......... .......... .......... 32% 184M 0s 2019-12-09T09:27:15.042 INFO:teuthology.orchestra.run.smithi035.stderr: 23950K .......... .......... .......... .......... .......... 32% 143M 0s 2019-12-09T09:27:15.042 INFO:teuthology.orchestra.run.smithi035.stderr: 24000K .......... .......... .......... .......... .......... 32% 169M 0s 2019-12-09T09:27:15.042 INFO:teuthology.orchestra.run.smithi035.stderr: 24050K .......... .......... .......... .......... .......... 32% 164M 0s 2019-12-09T09:27:15.042 INFO:teuthology.orchestra.run.smithi035.stderr: 24100K .......... .......... .......... .......... .......... 32% 185M 0s 2019-12-09T09:27:15.042 INFO:teuthology.orchestra.run.smithi035.stderr: 24150K .......... .......... .......... .......... .......... 32% 131M 0s 2019-12-09T09:27:15.042 INFO:teuthology.orchestra.run.smithi035.stderr: 24200K .......... .......... .......... .......... .......... 32% 186M 0s 2019-12-09T09:27:15.042 INFO:teuthology.orchestra.run.smithi035.stderr: 24250K .......... .......... .......... .......... .......... 33% 184M 0s 2019-12-09T09:27:15.043 INFO:teuthology.orchestra.run.smithi035.stderr: 24300K .......... .......... .......... .......... .......... 33% 161M 0s 2019-12-09T09:27:15.043 INFO:teuthology.orchestra.run.smithi035.stderr: 24350K .......... .......... .......... .......... .......... 33% 160M 0s 2019-12-09T09:27:15.043 INFO:teuthology.orchestra.run.smithi035.stderr: 24400K .......... .......... .......... .......... .......... 33% 171M 0s 2019-12-09T09:27:15.043 INFO:teuthology.orchestra.run.smithi035.stderr: 24450K .......... .......... .......... .......... .......... 33% 179M 0s 2019-12-09T09:27:15.043 INFO:teuthology.orchestra.run.smithi035.stderr: 24500K .......... .......... .......... .......... .......... 33% 195M 0s 2019-12-09T09:27:15.043 INFO:teuthology.orchestra.run.smithi035.stderr: 24550K .......... .......... .......... .......... .......... 33% 174M 0s 2019-12-09T09:27:15.043 INFO:teuthology.orchestra.run.smithi035.stderr: 24600K .......... .......... .......... .......... .......... 33% 199M 0s 2019-12-09T09:27:15.044 INFO:teuthology.orchestra.run.smithi035.stderr: 24650K .......... .......... .......... .......... .......... 33% 191M 0s 2019-12-09T09:27:15.044 INFO:teuthology.orchestra.run.smithi035.stderr: 24700K .......... .......... .......... .......... .......... 33% 193M 0s 2019-12-09T09:27:15.044 INFO:teuthology.orchestra.run.smithi035.stderr: 24750K .......... .......... .......... .......... .......... 33% 156M 0s 2019-12-09T09:27:15.044 INFO:teuthology.orchestra.run.smithi035.stderr: 24800K .......... .......... .......... .......... .......... 33% 189M 0s 2019-12-09T09:27:15.044 INFO:teuthology.orchestra.run.smithi035.stderr: 24850K .......... .......... .......... .......... .......... 33% 187M 0s 2019-12-09T09:27:15.044 INFO:teuthology.orchestra.run.smithi035.stderr: 24900K .......... .......... .......... .......... .......... 33% 171M 0s 2019-12-09T09:27:15.044 INFO:teuthology.orchestra.run.smithi035.stderr: 24950K .......... .......... .......... .......... .......... 33% 158M 0s 2019-12-09T09:27:15.044 INFO:teuthology.orchestra.run.smithi035.stderr: 25000K .......... .......... .......... .......... .......... 34% 176M 0s 2019-12-09T09:27:15.045 INFO:teuthology.orchestra.run.smithi035.stderr: 25050K .......... .......... .......... .......... .......... 34% 191M 0s 2019-12-09T09:27:15.045 INFO:teuthology.orchestra.run.smithi035.stderr: 25100K .......... .......... .......... .......... .......... 34% 193M 0s 2019-12-09T09:27:15.045 INFO:teuthology.orchestra.run.smithi035.stderr: 25150K .......... .......... .......... .......... .......... 34% 168M 0s 2019-12-09T09:27:15.045 INFO:teuthology.orchestra.run.smithi035.stderr: 25200K .......... .......... .......... .......... .......... 34% 173M 0s 2019-12-09T09:27:15.045 INFO:teuthology.orchestra.run.smithi035.stderr: 25250K .......... .......... .......... .......... .......... 34% 203M 0s 2019-12-09T09:27:15.045 INFO:teuthology.orchestra.run.smithi035.stderr: 25300K .......... .......... .......... .......... .......... 34% 187M 0s 2019-12-09T09:27:15.045 INFO:teuthology.orchestra.run.smithi035.stderr: 25350K .......... .......... .......... .......... .......... 34% 164M 0s 2019-12-09T09:27:15.046 INFO:teuthology.orchestra.run.smithi035.stderr: 25400K .......... .......... .......... .......... .......... 34% 203M 0s 2019-12-09T09:27:15.046 INFO:teuthology.orchestra.run.smithi035.stderr: 25450K .......... .......... .......... .......... .......... 34% 198M 0s 2019-12-09T09:27:15.046 INFO:teuthology.orchestra.run.smithi035.stderr: 25500K .......... .......... .......... .......... .......... 34% 201M 0s 2019-12-09T09:27:15.046 INFO:teuthology.orchestra.run.smithi035.stderr: 25550K .......... .......... .......... .......... .......... 34% 155M 0s 2019-12-09T09:27:15.046 INFO:teuthology.orchestra.run.smithi035.stderr: 25600K .......... .......... .......... .......... .......... 34% 196M 0s 2019-12-09T09:27:15.046 INFO:teuthology.orchestra.run.smithi035.stderr: 25650K .......... .......... .......... .......... .......... 34% 196M 0s 2019-12-09T09:27:15.046 INFO:teuthology.orchestra.run.smithi035.stderr: 25700K .......... .......... .......... .......... .......... 35% 184M 0s 2019-12-09T09:27:15.047 INFO:teuthology.orchestra.run.smithi035.stderr: 25750K .......... .......... .......... .......... .......... 35% 177M 0s 2019-12-09T09:27:15.047 INFO:teuthology.orchestra.run.smithi035.stderr: 25800K .......... .......... .......... .......... .......... 35% 188M 0s 2019-12-09T09:27:15.047 INFO:teuthology.orchestra.run.smithi035.stderr: 25850K .......... .......... .......... .......... .......... 35% 176M 0s 2019-12-09T09:27:15.047 INFO:teuthology.orchestra.run.smithi035.stderr: 25900K .......... .......... .......... .......... .......... 35% 190M 0s 2019-12-09T09:27:15.047 INFO:teuthology.orchestra.run.smithi035.stderr: 25950K .......... .......... .......... .......... .......... 35% 151M 0s 2019-12-09T09:27:15.047 INFO:teuthology.orchestra.run.smithi035.stderr: 26000K .......... .......... .......... .......... .......... 35% 182M 0s 2019-12-09T09:27:15.047 INFO:teuthology.orchestra.run.smithi035.stderr: 26050K .......... .......... .......... .......... .......... 35% 203M 0s 2019-12-09T09:27:15.047 INFO:teuthology.orchestra.run.smithi035.stderr: 26100K .......... .......... .......... .......... .......... 35% 206M 0s 2019-12-09T09:27:15.048 INFO:teuthology.orchestra.run.smithi035.stderr: 26150K .......... .......... .......... .......... .......... 35% 159M 0s 2019-12-09T09:27:15.048 INFO:teuthology.orchestra.run.smithi035.stderr: 26200K .......... .......... .......... .......... .......... 35% 196M 0s 2019-12-09T09:27:15.048 INFO:teuthology.orchestra.run.smithi035.stderr: 26250K .......... .......... .......... .......... .......... 35% 201M 0s 2019-12-09T09:27:15.048 INFO:teuthology.orchestra.run.smithi035.stderr: 26300K .......... .......... .......... .......... .......... 35% 196M 0s 2019-12-09T09:27:15.048 INFO:teuthology.orchestra.run.smithi035.stderr: 26350K .......... .......... .......... .......... .......... 35% 179M 0s 2019-12-09T09:27:15.048 INFO:teuthology.orchestra.run.smithi035.stderr: 26400K .......... .......... .......... .......... .......... 35% 204M 0s 2019-12-09T09:27:15.048 INFO:teuthology.orchestra.run.smithi035.stderr: 26450K .......... .......... .......... .......... .......... 36% 196M 0s 2019-12-09T09:27:15.049 INFO:teuthology.orchestra.run.smithi035.stderr: 26500K .......... .......... .......... .......... .......... 36% 190M 0s 2019-12-09T09:27:15.049 INFO:teuthology.orchestra.run.smithi035.stderr: 26550K .......... .......... .......... .......... .......... 36% 162M 0s 2019-12-09T09:27:15.049 INFO:teuthology.orchestra.run.smithi035.stderr: 26600K .......... .......... .......... .......... .......... 36% 192M 0s 2019-12-09T09:27:15.049 INFO:teuthology.orchestra.run.smithi035.stderr: 26650K .......... .......... .......... .......... .......... 36% 183M 0s 2019-12-09T09:27:15.049 INFO:teuthology.orchestra.run.smithi035.stderr: 26700K .......... .......... .......... .......... .......... 36% 185M 0s 2019-12-09T09:27:15.049 INFO:teuthology.orchestra.run.smithi035.stderr: 26750K .......... .......... .......... .......... .......... 36% 165M 0s 2019-12-09T09:27:15.049 INFO:teuthology.orchestra.run.smithi035.stderr: 26800K .......... .......... .......... .......... .......... 36% 193M 0s 2019-12-09T09:27:15.050 INFO:teuthology.orchestra.run.smithi035.stderr: 26850K .......... .......... .......... .......... .......... 36% 204M 0s 2019-12-09T09:27:15.050 INFO:teuthology.orchestra.run.smithi035.stderr: 26900K .......... .......... .......... .......... .......... 36% 176M 0s 2019-12-09T09:27:15.050 INFO:teuthology.orchestra.run.smithi035.stderr: 26950K .......... .......... .......... .......... .......... 36% 154M 0s 2019-12-09T09:27:15.050 INFO:teuthology.orchestra.run.smithi035.stderr: 27000K .......... .......... .......... .......... .......... 36% 204M 0s 2019-12-09T09:27:15.050 INFO:teuthology.orchestra.run.smithi035.stderr: 27050K .......... .......... .......... .......... .......... 36% 205M 0s 2019-12-09T09:27:15.050 INFO:teuthology.orchestra.run.smithi035.stderr: 27100K .......... .......... .......... .......... .......... 36% 188M 0s 2019-12-09T09:27:15.050 INFO:teuthology.orchestra.run.smithi035.stderr: 27150K .......... .......... .......... .......... .......... 36% 150M 0s 2019-12-09T09:27:15.050 INFO:teuthology.orchestra.run.smithi035.stderr: 27200K .......... .......... .......... .......... .......... 37% 182M 0s 2019-12-09T09:27:15.051 INFO:teuthology.orchestra.run.smithi035.stderr: 27250K .......... .......... .......... .......... .......... 37% 193M 0s 2019-12-09T09:27:15.051 INFO:teuthology.orchestra.run.smithi035.stderr: 27300K .......... .......... .......... .......... .......... 37% 180M 0s 2019-12-09T09:27:15.051 INFO:teuthology.orchestra.run.smithi035.stderr: 27350K .......... .......... .......... .......... .......... 37% 167M 0s 2019-12-09T09:27:15.051 INFO:teuthology.orchestra.run.smithi035.stderr: 27400K .......... .......... .......... .......... .......... 37% 181M 0s 2019-12-09T09:27:15.051 INFO:teuthology.orchestra.run.smithi035.stderr: 27450K .......... .......... .......... .......... .......... 37% 182M 0s 2019-12-09T09:27:15.051 INFO:teuthology.orchestra.run.smithi035.stderr: 27500K .......... .......... .......... .......... .......... 37% 182M 0s 2019-12-09T09:27:15.051 INFO:teuthology.orchestra.run.smithi035.stderr: 27550K .......... .......... .......... .......... .......... 37% 158M 0s 2019-12-09T09:27:15.052 INFO:teuthology.orchestra.run.smithi035.stderr: 27600K .......... .......... .......... .......... .......... 37% 191M 0s 2019-12-09T09:27:15.052 INFO:teuthology.orchestra.run.smithi035.stderr: 27650K .......... .......... .......... .......... .......... 37% 188M 0s 2019-12-09T09:27:15.052 INFO:teuthology.orchestra.run.smithi035.stderr: 27700K .......... .......... .......... .......... .......... 37% 196M 0s 2019-12-09T09:27:15.052 INFO:teuthology.orchestra.run.smithi035.stderr: 27750K .......... .......... .......... .......... .......... 37% 181M 0s 2019-12-09T09:27:15.052 INFO:teuthology.orchestra.run.smithi035.stderr: 27800K .......... .......... .......... .......... .......... 37% 207M 0s 2019-12-09T09:27:15.052 INFO:teuthology.orchestra.run.smithi035.stderr: 27850K .......... .......... .......... .......... .......... 37% 195M 0s 2019-12-09T09:27:15.052 INFO:teuthology.orchestra.run.smithi035.stderr: 27900K .......... .......... .......... .......... .......... 37% 200M 0s 2019-12-09T09:27:15.053 INFO:teuthology.orchestra.run.smithi035.stderr: 27950K .......... .......... .......... .......... .......... 38% 162M 0s 2019-12-09T09:27:15.053 INFO:teuthology.orchestra.run.smithi035.stderr: 28000K .......... .......... .......... .......... .......... 38% 197M 0s 2019-12-09T09:27:15.053 INFO:teuthology.orchestra.run.smithi035.stderr: 28050K .......... .......... .......... .......... .......... 38% 175M 0s 2019-12-09T09:27:15.053 INFO:teuthology.orchestra.run.smithi035.stderr: 28100K .......... .......... .......... .......... .......... 38% 195M 0s 2019-12-09T09:27:15.053 INFO:teuthology.orchestra.run.smithi035.stderr: 28150K .......... .......... .......... .......... .......... 38% 166M 0s 2019-12-09T09:27:15.053 INFO:teuthology.orchestra.run.smithi035.stderr: 28200K .......... .......... .......... .......... .......... 38% 194M 0s 2019-12-09T09:27:15.053 INFO:teuthology.orchestra.run.smithi035.stderr: 28250K .......... .......... .......... .......... .......... 38% 212M 0s 2019-12-09T09:27:15.053 INFO:teuthology.orchestra.run.smithi035.stderr: 28300K .......... .......... .......... .......... .......... 38% 209M 0s 2019-12-09T09:27:15.054 INFO:teuthology.orchestra.run.smithi035.stderr: 28350K .......... .......... .......... .......... .......... 38% 156M 0s 2019-12-09T09:27:15.054 INFO:teuthology.orchestra.run.smithi035.stderr: 28400K .......... .......... .......... .......... .......... 38% 188M 0s 2019-12-09T09:27:15.054 INFO:teuthology.orchestra.run.smithi035.stderr: 28450K .......... .......... .......... .......... .......... 38% 206M 0s 2019-12-09T09:27:15.054 INFO:teuthology.orchestra.run.smithi035.stderr: 28500K .......... .......... .......... .......... .......... 38% 193M 0s 2019-12-09T09:27:15.054 INFO:teuthology.orchestra.run.smithi035.stderr: 28550K .......... .......... .......... .......... .......... 38% 185M 0s 2019-12-09T09:27:15.054 INFO:teuthology.orchestra.run.smithi035.stderr: 28600K .......... .......... .......... .......... .......... 38% 195M 0s 2019-12-09T09:27:15.054 INFO:teuthology.orchestra.run.smithi035.stderr: 28650K .......... .......... .......... .......... .......... 39% 182M 0s 2019-12-09T09:27:15.055 INFO:teuthology.orchestra.run.smithi035.stderr: 28700K .......... .......... .......... .......... .......... 39% 207M 0s 2019-12-09T09:27:15.055 INFO:teuthology.orchestra.run.smithi035.stderr: 28750K .......... .......... .......... .......... .......... 39% 170M 0s 2019-12-09T09:27:15.055 INFO:teuthology.orchestra.run.smithi035.stderr: 28800K .......... .......... .......... .......... .......... 39% 178M 0s 2019-12-09T09:27:15.055 INFO:teuthology.orchestra.run.smithi035.stderr: 28850K .......... .......... .......... .......... .......... 39% 189M 0s 2019-12-09T09:27:15.055 INFO:teuthology.orchestra.run.smithi035.stderr: 28900K .......... .......... .......... .......... .......... 39% 208M 0s 2019-12-09T09:27:15.055 INFO:teuthology.orchestra.run.smithi035.stderr: 28950K .......... .......... .......... .......... .......... 39% 176M 0s 2019-12-09T09:27:15.055 INFO:teuthology.orchestra.run.smithi035.stderr: 29000K .......... .......... .......... .......... .......... 39% 194M 0s 2019-12-09T09:27:15.056 INFO:teuthology.orchestra.run.smithi035.stderr: 29050K .......... .......... .......... .......... .......... 39% 207M 0s 2019-12-09T09:27:15.056 INFO:teuthology.orchestra.run.smithi035.stderr: 29100K .......... .......... .......... .......... .......... 39% 209M 0s 2019-12-09T09:27:15.056 INFO:teuthology.orchestra.run.smithi035.stderr: 29150K .......... .......... .......... .......... .......... 39% 173M 0s 2019-12-09T09:27:15.056 INFO:teuthology.orchestra.run.smithi035.stderr: 29200K .......... .......... .......... .......... .......... 39% 203M 0s 2019-12-09T09:27:15.056 INFO:teuthology.orchestra.run.smithi035.stderr: 29250K .......... .......... .......... .......... .......... 39% 200M 0s 2019-12-09T09:27:15.056 INFO:teuthology.orchestra.run.smithi035.stderr: 29300K .......... .......... .......... .......... .......... 39% 205M 0s 2019-12-09T09:27:15.056 INFO:teuthology.orchestra.run.smithi035.stderr: 29350K .......... .......... .......... .......... .......... 39% 174M 0s 2019-12-09T09:27:15.057 INFO:teuthology.orchestra.run.smithi035.stderr: 29400K .......... .......... .......... .......... .......... 40% 206M 0s 2019-12-09T09:27:15.057 INFO:teuthology.orchestra.run.smithi035.stderr: 29450K .......... .......... .......... .......... .......... 40% 189M 0s 2019-12-09T09:27:15.057 INFO:teuthology.orchestra.run.smithi035.stderr: 29500K .......... .......... .......... .......... .......... 40% 193M 0s 2019-12-09T09:27:15.057 INFO:teuthology.orchestra.run.smithi035.stderr: 29550K .......... .......... .......... .......... .......... 40% 157M 0s 2019-12-09T09:27:15.057 INFO:teuthology.orchestra.run.smithi035.stderr: 29600K .......... .......... .......... .......... .......... 40% 199M 0s 2019-12-09T09:27:15.057 INFO:teuthology.orchestra.run.smithi035.stderr: 29650K .......... .......... .......... .......... .......... 40% 194M 0s 2019-12-09T09:27:15.057 INFO:teuthology.orchestra.run.smithi035.stderr: 29700K .......... .......... .......... .......... .......... 40% 224M 0s 2019-12-09T09:27:15.058 INFO:teuthology.orchestra.run.smithi035.stderr: 29750K .......... .......... .......... .......... .......... 40% 169M 0s 2019-12-09T09:27:15.058 INFO:teuthology.orchestra.run.smithi035.stderr: 29800K .......... .......... .......... .......... .......... 40% 174M 0s 2019-12-09T09:27:15.058 INFO:teuthology.orchestra.run.smithi035.stderr: 29850K .......... .......... .......... .......... .......... 40% 187M 0s 2019-12-09T09:27:15.058 INFO:teuthology.orchestra.run.smithi035.stderr: 29900K .......... .......... .......... .......... .......... 40% 198M 0s 2019-12-09T09:27:15.058 INFO:teuthology.orchestra.run.smithi035.stderr: 29950K .......... .......... .......... .......... .......... 40% 151M 0s 2019-12-09T09:27:15.058 INFO:teuthology.orchestra.run.smithi035.stderr: 30000K .......... .......... .......... .......... .......... 40% 201M 0s 2019-12-09T09:27:15.058 INFO:teuthology.orchestra.run.smithi035.stderr: 30050K .......... .......... .......... .......... .......... 40% 186M 0s 2019-12-09T09:27:15.059 INFO:teuthology.orchestra.run.smithi035.stderr: 30100K .......... .......... .......... .......... .......... 40% 197M 0s 2019-12-09T09:27:15.059 INFO:teuthology.orchestra.run.smithi035.stderr: 30150K .......... .......... .......... .......... .......... 41% 175M 0s 2019-12-09T09:27:15.059 INFO:teuthology.orchestra.run.smithi035.stderr: 30200K .......... .......... .......... .......... .......... 41% 207M 0s 2019-12-09T09:27:15.059 INFO:teuthology.orchestra.run.smithi035.stderr: 30250K .......... .......... .......... .......... .......... 41% 192M 0s 2019-12-09T09:27:15.059 INFO:teuthology.orchestra.run.smithi035.stderr: 30300K .......... .......... .......... .......... .......... 41% 209M 0s 2019-12-09T09:27:15.060 INFO:teuthology.orchestra.run.smithi035.stderr: 30350K .......... .......... .......... .......... .......... 41% 152M 0s 2019-12-09T09:27:15.060 INFO:teuthology.orchestra.run.smithi035.stderr: 30400K .......... .......... .......... .......... .......... 41% 206M 0s 2019-12-09T09:27:15.060 INFO:teuthology.orchestra.run.smithi035.stderr: 30450K .......... .......... .......... .......... .......... 41% 209M 0s 2019-12-09T09:27:15.060 INFO:teuthology.orchestra.run.smithi035.stderr: 30500K .......... .......... .......... .......... .......... 41% 223M 0s 2019-12-09T09:27:15.060 INFO:teuthology.orchestra.run.smithi035.stderr: 30550K .......... .......... .......... .......... .......... 41% 168M 0s 2019-12-09T09:27:15.060 INFO:teuthology.orchestra.run.smithi035.stderr: 30600K .......... .......... .......... .......... .......... 41% 201M 0s 2019-12-09T09:27:15.060 INFO:teuthology.orchestra.run.smithi035.stderr: 30650K .......... .......... .......... .......... .......... 41% 186M 0s 2019-12-09T09:27:15.061 INFO:teuthology.orchestra.run.smithi035.stderr: 30700K .......... .......... .......... .......... .......... 41% 219M 0s 2019-12-09T09:27:15.061 INFO:teuthology.orchestra.run.smithi035.stderr: 30750K .......... .......... .......... .......... .......... 41% 172M 0s 2019-12-09T09:27:15.061 INFO:teuthology.orchestra.run.smithi035.stderr: 30800K .......... .......... .......... .......... .......... 41% 192M 0s 2019-12-09T09:27:15.061 INFO:teuthology.orchestra.run.smithi035.stderr: 30850K .......... .......... .......... .......... .......... 42% 181M 0s 2019-12-09T09:27:15.061 INFO:teuthology.orchestra.run.smithi035.stderr: 30900K .......... .......... .......... .......... .......... 42% 192M 0s 2019-12-09T09:27:15.061 INFO:teuthology.orchestra.run.smithi035.stderr: 30950K .......... .......... .......... .......... .......... 42% 160M 0s 2019-12-09T09:27:15.061 INFO:teuthology.orchestra.run.smithi035.stderr: 31000K .......... .......... .......... .......... .......... 42% 202M 0s 2019-12-09T09:27:15.061 INFO:teuthology.orchestra.run.smithi035.stderr: 31050K .......... .......... .......... .......... .......... 42% 210M 0s 2019-12-09T09:27:15.062 INFO:teuthology.orchestra.run.smithi035.stderr: 31100K .......... .......... .......... .......... .......... 42% 194M 0s 2019-12-09T09:27:15.062 INFO:teuthology.orchestra.run.smithi035.stderr: 31150K .......... .......... .......... .......... .......... 42% 166M 0s 2019-12-09T09:27:15.062 INFO:teuthology.orchestra.run.smithi035.stderr: 31200K .......... .......... .......... .......... .......... 42% 195M 0s 2019-12-09T09:27:15.062 INFO:teuthology.orchestra.run.smithi035.stderr: 31250K .......... .......... .......... .......... .......... 42% 187M 0s 2019-12-09T09:27:15.062 INFO:teuthology.orchestra.run.smithi035.stderr: 31300K .......... .......... .......... .......... .......... 42% 207M 0s 2019-12-09T09:27:15.062 INFO:teuthology.orchestra.run.smithi035.stderr: 31350K .......... .......... .......... .......... .......... 42% 168M 0s 2019-12-09T09:27:15.062 INFO:teuthology.orchestra.run.smithi035.stderr: 31400K .......... .......... .......... .......... .......... 42% 215M 0s 2019-12-09T09:27:15.063 INFO:teuthology.orchestra.run.smithi035.stderr: 31450K .......... .......... .......... .......... .......... 42% 215M 0s 2019-12-09T09:27:15.063 INFO:teuthology.orchestra.run.smithi035.stderr: 31500K .......... .......... .......... .......... .......... 42% 196M 0s 2019-12-09T09:27:15.063 INFO:teuthology.orchestra.run.smithi035.stderr: 31550K .......... .......... .......... .......... .......... 42% 176M 0s 2019-12-09T09:27:15.063 INFO:teuthology.orchestra.run.smithi035.stderr: 31600K .......... .......... .......... .......... .......... 43% 227M 0s 2019-12-09T09:27:15.063 INFO:teuthology.orchestra.run.smithi035.stderr: 31650K .......... .......... .......... .......... .......... 43% 220M 0s 2019-12-09T09:27:15.063 INFO:teuthology.orchestra.run.smithi035.stderr: 31700K .......... .......... .......... .......... .......... 43% 215M 0s 2019-12-09T09:27:15.063 INFO:teuthology.orchestra.run.smithi035.stderr: 31750K .......... .......... .......... .......... .......... 43% 169M 0s 2019-12-09T09:27:15.064 INFO:teuthology.orchestra.run.smithi035.stderr: 31800K .......... .......... .......... .......... .......... 43% 226M 0s 2019-12-09T09:27:15.064 INFO:teuthology.orchestra.run.smithi035.stderr: 31850K .......... .......... .......... .......... .......... 43% 208M 0s 2019-12-09T09:27:15.064 INFO:teuthology.orchestra.run.smithi035.stderr: 31900K .......... .......... .......... .......... .......... 43% 204M 0s 2019-12-09T09:27:15.064 INFO:teuthology.orchestra.run.smithi035.stderr: 31950K .......... .......... .......... .......... .......... 43% 187M 0s 2019-12-09T09:27:15.064 INFO:teuthology.orchestra.run.smithi035.stderr: 32000K .......... .......... .......... .......... .......... 43% 220M 0s 2019-12-09T09:27:15.064 INFO:teuthology.orchestra.run.smithi035.stderr: 32050K .......... .......... .......... .......... .......... 43% 207M 0s 2019-12-09T09:27:15.064 INFO:teuthology.orchestra.run.smithi035.stderr: 32100K .......... .......... .......... .......... .......... 43% 223M 0s 2019-12-09T09:27:15.065 INFO:teuthology.orchestra.run.smithi035.stderr: 32150K .......... .......... .......... .......... .......... 43% 168M 0s 2019-12-09T09:27:15.065 INFO:teuthology.orchestra.run.smithi035.stderr: 32200K .......... .......... .......... .......... .......... 43% 213M 0s 2019-12-09T09:27:15.065 INFO:teuthology.orchestra.run.smithi035.stderr: 32250K .......... .......... .......... .......... .......... 43% 202M 0s 2019-12-09T09:27:15.065 INFO:teuthology.orchestra.run.smithi035.stderr: 32300K .......... .......... .......... .......... .......... 43% 217M 0s 2019-12-09T09:27:15.065 INFO:teuthology.orchestra.run.smithi035.stderr: 32350K .......... .......... .......... .......... .......... 44% 182M 0s 2019-12-09T09:27:15.065 INFO:teuthology.orchestra.run.smithi035.stderr: 32400K .......... .......... .......... .......... .......... 44% 225M 0s 2019-12-09T09:27:15.066 INFO:teuthology.orchestra.run.smithi035.stderr: 32450K .......... .......... .......... .......... .......... 44% 196M 0s 2019-12-09T09:27:15.066 INFO:teuthology.orchestra.run.smithi035.stderr: 32500K .......... .......... .......... .......... .......... 44% 213M 0s 2019-12-09T09:27:15.066 INFO:teuthology.orchestra.run.smithi035.stderr: 32550K .......... .......... .......... .......... .......... 44% 164M 0s 2019-12-09T09:27:15.066 INFO:teuthology.orchestra.run.smithi035.stderr: 32600K .......... .......... .......... .......... .......... 44% 184M 0s 2019-12-09T09:27:15.066 INFO:teuthology.orchestra.run.smithi035.stderr: 32650K .......... .......... .......... .......... .......... 44% 192M 0s 2019-12-09T09:27:15.066 INFO:teuthology.orchestra.run.smithi035.stderr: 32700K .......... .......... .......... .......... .......... 44% 207M 0s 2019-12-09T09:27:15.067 INFO:teuthology.orchestra.run.smithi035.stderr: 32750K .......... .......... .......... .......... .......... 44% 142M 0s 2019-12-09T09:27:15.067 INFO:teuthology.orchestra.run.smithi035.stderr: 32800K .......... .......... .......... .......... .......... 44% 223M 0s 2019-12-09T09:27:15.067 INFO:teuthology.orchestra.run.smithi035.stderr: 32850K .......... .......... .......... .......... .......... 44% 195M 0s 2019-12-09T09:27:15.067 INFO:teuthology.orchestra.run.smithi035.stderr: 32900K .......... .......... .......... .......... .......... 44% 204M 0s 2019-12-09T09:27:15.067 INFO:teuthology.orchestra.run.smithi035.stderr: 32950K .......... .......... .......... .......... .......... 44% 185M 0s 2019-12-09T09:27:15.067 INFO:teuthology.orchestra.run.smithi035.stderr: 33000K .......... .......... .......... .......... .......... 44% 212M 0s 2019-12-09T09:27:15.067 INFO:teuthology.orchestra.run.smithi035.stderr: 33050K .......... .......... .......... .......... .......... 44% 201M 0s 2019-12-09T09:27:15.068 INFO:teuthology.orchestra.run.smithi035.stderr: 33100K .......... .......... .......... .......... .......... 45% 223M 0s 2019-12-09T09:27:15.068 INFO:teuthology.orchestra.run.smithi035.stderr: 33150K .......... .......... .......... .......... .......... 45% 170M 0s 2019-12-09T09:27:15.068 INFO:teuthology.orchestra.run.smithi035.stderr: 33200K .......... .......... .......... .......... .......... 45% 225M 0s 2019-12-09T09:27:15.068 INFO:teuthology.orchestra.run.smithi035.stderr: 33250K .......... .......... .......... .......... .......... 45% 208M 0s 2019-12-09T09:27:15.068 INFO:teuthology.orchestra.run.smithi035.stderr: 33300K .......... .......... .......... .......... .......... 45% 209M 0s 2019-12-09T09:27:15.068 INFO:teuthology.orchestra.run.smithi035.stderr: 33350K .......... .......... .......... .......... .......... 45% 180M 0s 2019-12-09T09:27:15.068 INFO:teuthology.orchestra.run.smithi035.stderr: 33400K .......... .......... .......... .......... .......... 45% 219M 0s 2019-12-09T09:27:15.069 INFO:teuthology.orchestra.run.smithi035.stderr: 33450K .......... .......... .......... .......... .......... 45% 221M 0s 2019-12-09T09:27:15.069 INFO:teuthology.orchestra.run.smithi035.stderr: 33500K .......... .......... .......... .......... .......... 45% 233M 0s 2019-12-09T09:27:15.069 INFO:teuthology.orchestra.run.smithi035.stderr: 33550K .......... .......... .......... .......... .......... 45% 185M 0s 2019-12-09T09:27:15.069 INFO:teuthology.orchestra.run.smithi035.stderr: 33600K .......... .......... .......... .......... .......... 45% 232M 0s 2019-12-09T09:27:15.069 INFO:teuthology.orchestra.run.smithi035.stderr: 33650K .......... .......... .......... .......... .......... 45% 225M 0s 2019-12-09T09:27:15.069 INFO:teuthology.orchestra.run.smithi035.stderr: 33700K .......... .......... .......... .......... .......... 45% 231M 0s 2019-12-09T09:27:15.069 INFO:teuthology.orchestra.run.smithi035.stderr: 33750K .......... .......... .......... .......... .......... 45% 211M 0s 2019-12-09T09:27:15.070 INFO:teuthology.orchestra.run.smithi035.stderr: 33800K .......... .......... .......... .......... .......... 46% 224M 0s 2019-12-09T09:27:15.070 INFO:teuthology.orchestra.run.smithi035.stderr: 33850K .......... .......... .......... .......... .......... 46% 213M 0s 2019-12-09T09:27:15.070 INFO:teuthology.orchestra.run.smithi035.stderr: 33900K .......... .......... .......... .......... .......... 46% 229M 0s 2019-12-09T09:27:15.070 INFO:teuthology.orchestra.run.smithi035.stderr: 33950K .......... .......... .......... .......... .......... 46% 207M 0s 2019-12-09T09:27:15.070 INFO:teuthology.orchestra.run.smithi035.stderr: 34000K .......... .......... .......... .......... .......... 46% 204M 0s 2019-12-09T09:27:15.070 INFO:teuthology.orchestra.run.smithi035.stderr: 34050K .......... .......... .......... .......... .......... 46% 227M 0s 2019-12-09T09:27:15.070 INFO:teuthology.orchestra.run.smithi035.stderr: 34100K .......... .......... .......... .......... .......... 46% 232M 0s 2019-12-09T09:27:15.070 INFO:teuthology.orchestra.run.smithi035.stderr: 34150K .......... .......... .......... .......... .......... 46% 173M 0s 2019-12-09T09:27:15.071 INFO:teuthology.orchestra.run.smithi035.stderr: 34200K .......... .......... .......... .......... .......... 46% 219M 0s 2019-12-09T09:27:15.071 INFO:teuthology.orchestra.run.smithi035.stderr: 34250K .......... .......... .......... .......... .......... 46% 220M 0s 2019-12-09T09:27:15.071 INFO:teuthology.orchestra.run.smithi035.stderr: 34300K .......... .......... .......... .......... .......... 46% 233M 0s 2019-12-09T09:27:15.071 INFO:teuthology.orchestra.run.smithi035.stderr: 34350K .......... .......... .......... .......... .......... 46% 180M 0s 2019-12-09T09:27:15.071 INFO:teuthology.orchestra.run.smithi035.stderr: 34400K .......... .......... .......... .......... .......... 46% 220M 0s 2019-12-09T09:27:15.071 INFO:teuthology.orchestra.run.smithi035.stderr: 34450K .......... .......... .......... .......... .......... 46% 184M 0s 2019-12-09T09:27:15.071 INFO:teuthology.orchestra.run.smithi035.stderr: 34500K .......... .......... .......... .......... .......... 46% 216M 0s 2019-12-09T09:27:15.072 INFO:teuthology.orchestra.run.smithi035.stderr: 34550K .......... .......... .......... .......... .......... 47% 174M 0s 2019-12-09T09:27:15.072 INFO:teuthology.orchestra.run.smithi035.stderr: 34600K .......... .......... .......... .......... .......... 47% 219M 0s 2019-12-09T09:27:15.072 INFO:teuthology.orchestra.run.smithi035.stderr: 34650K .......... .......... .......... .......... .......... 47% 205M 0s 2019-12-09T09:27:15.072 INFO:teuthology.orchestra.run.smithi035.stderr: 34700K .......... .......... .......... .......... .......... 47% 228M 0s 2019-12-09T09:27:15.072 INFO:teuthology.orchestra.run.smithi035.stderr: 34750K .......... .......... .......... .......... .......... 47% 175M 0s 2019-12-09T09:27:15.072 INFO:teuthology.orchestra.run.smithi035.stderr: 34800K .......... .......... .......... .......... .......... 47% 214M 0s 2019-12-09T09:27:15.072 INFO:teuthology.orchestra.run.smithi035.stderr: 34850K .......... .......... .......... .......... .......... 47% 215M 0s 2019-12-09T09:27:15.073 INFO:teuthology.orchestra.run.smithi035.stderr: 34900K .......... .......... .......... .......... .......... 47% 217M 0s 2019-12-09T09:27:15.073 INFO:teuthology.orchestra.run.smithi035.stderr: 34950K .......... .......... .......... .......... .......... 47% 181M 0s 2019-12-09T09:27:15.073 INFO:teuthology.orchestra.run.smithi035.stderr: 35000K .......... .......... .......... .......... .......... 47% 218M 0s 2019-12-09T09:27:15.073 INFO:teuthology.orchestra.run.smithi035.stderr: 35050K .......... .......... .......... .......... .......... 47% 238M 0s 2019-12-09T09:27:15.073 INFO:teuthology.orchestra.run.smithi035.stderr: 35100K .......... .......... .......... .......... .......... 47% 221M 0s 2019-12-09T09:27:15.073 INFO:teuthology.orchestra.run.smithi035.stderr: 35150K .......... .......... .......... .......... .......... 47% 203M 0s 2019-12-09T09:27:15.073 INFO:teuthology.orchestra.run.smithi035.stderr: 35200K .......... .......... .......... .......... .......... 47% 214M 0s 2019-12-09T09:27:15.074 INFO:teuthology.orchestra.run.smithi035.stderr: 35250K .......... .......... .......... .......... .......... 47% 240M 0s 2019-12-09T09:27:15.074 INFO:teuthology.orchestra.run.smithi035.stderr: 35300K .......... .......... .......... .......... .......... 48% 206M 0s 2019-12-09T09:27:15.074 INFO:teuthology.orchestra.run.smithi035.stderr: 35350K .......... .......... .......... .......... .......... 48% 202M 0s 2019-12-09T09:27:15.074 INFO:teuthology.orchestra.run.smithi035.stderr: 35400K .......... .......... .......... .......... .......... 48% 229M 0s 2019-12-09T09:27:15.074 INFO:teuthology.orchestra.run.smithi035.stderr: 35450K .......... .......... .......... .......... .......... 48% 206M 0s 2019-12-09T09:27:15.074 INFO:teuthology.orchestra.run.smithi035.stderr: 35500K .......... .......... .......... .......... .......... 48% 212M 0s 2019-12-09T09:27:15.074 INFO:teuthology.orchestra.run.smithi035.stderr: 35550K .......... .......... .......... .......... .......... 48% 204M 0s 2019-12-09T09:27:15.075 INFO:teuthology.orchestra.run.smithi035.stderr: 35600K .......... .......... .......... .......... .......... 48% 237M 0s 2019-12-09T09:27:15.075 INFO:teuthology.orchestra.run.smithi035.stderr: 35650K .......... .......... .......... .......... .......... 48% 222M 0s 2019-12-09T09:27:15.075 INFO:teuthology.orchestra.run.smithi035.stderr: 35700K .......... .......... .......... .......... .......... 48% 205M 0s 2019-12-09T09:27:15.075 INFO:teuthology.orchestra.run.smithi035.stderr: 35750K .......... .......... .......... .......... .......... 48% 188M 0s 2019-12-09T09:27:15.075 INFO:teuthology.orchestra.run.smithi035.stderr: 35800K .......... .......... .......... .......... .......... 48% 225M 0s 2019-12-09T09:27:15.075 INFO:teuthology.orchestra.run.smithi035.stderr: 35850K .......... .......... .......... .......... .......... 48% 217M 0s 2019-12-09T09:27:15.075 INFO:teuthology.orchestra.run.smithi035.stderr: 35900K .......... .......... .......... .......... .......... 48% 225M 0s 2019-12-09T09:27:15.075 INFO:teuthology.orchestra.run.smithi035.stderr: 35950K .......... .......... .......... .......... .......... 48% 187M 0s 2019-12-09T09:27:15.076 INFO:teuthology.orchestra.run.smithi035.stderr: 36000K .......... .......... .......... .......... .......... 49% 228M 0s 2019-12-09T09:27:15.076 INFO:teuthology.orchestra.run.smithi035.stderr: 36050K .......... .......... .......... .......... .......... 49% 216M 0s 2019-12-09T09:27:15.076 INFO:teuthology.orchestra.run.smithi035.stderr: 36100K .......... .......... .......... .......... .......... 49% 242M 0s 2019-12-09T09:27:15.076 INFO:teuthology.orchestra.run.smithi035.stderr: 36150K .......... .......... .......... .......... .......... 49% 190M 0s 2019-12-09T09:27:15.076 INFO:teuthology.orchestra.run.smithi035.stderr: 36200K .......... .......... .......... .......... .......... 49% 231M 0s 2019-12-09T09:27:15.076 INFO:teuthology.orchestra.run.smithi035.stderr: 36250K .......... .......... .......... .......... .......... 49% 209M 0s 2019-12-09T09:27:15.076 INFO:teuthology.orchestra.run.smithi035.stderr: 36300K .......... .......... .......... .......... .......... 49% 237M 0s 2019-12-09T09:27:15.077 INFO:teuthology.orchestra.run.smithi035.stderr: 36350K .......... .......... .......... .......... .......... 49% 177M 0s 2019-12-09T09:27:15.077 INFO:teuthology.orchestra.run.smithi035.stderr: 36400K .......... .......... .......... .......... .......... 49% 216M 0s 2019-12-09T09:27:15.077 INFO:teuthology.orchestra.run.smithi035.stderr: 36450K .......... .......... .......... .......... .......... 49% 228M 0s 2019-12-09T09:27:15.077 INFO:teuthology.orchestra.run.smithi035.stderr: 36500K .......... .......... .......... .......... .......... 49% 222M 0s 2019-12-09T09:27:15.077 INFO:teuthology.orchestra.run.smithi035.stderr: 36550K .......... .......... .......... .......... .......... 49% 178M 0s 2019-12-09T09:27:15.077 INFO:teuthology.orchestra.run.smithi035.stderr: 36600K .......... .......... .......... .......... .......... 49% 203M 0s 2019-12-09T09:27:15.077 INFO:teuthology.orchestra.run.smithi035.stderr: 36650K .......... .......... .......... .......... .......... 49% 204M 0s 2019-12-09T09:27:15.078 INFO:teuthology.orchestra.run.smithi035.stderr: 36700K .......... .......... .......... .......... .......... 49% 220M 0s 2019-12-09T09:27:15.078 INFO:teuthology.orchestra.run.smithi035.stderr: 36750K .......... .......... .......... .......... .......... 50% 172M 0s 2019-12-09T09:27:15.078 INFO:teuthology.orchestra.run.smithi035.stderr: 36800K .......... .......... .......... .......... .......... 50% 210M 0s 2019-12-09T09:27:15.078 INFO:teuthology.orchestra.run.smithi035.stderr: 36850K .......... .......... .......... .......... .......... 50% 211M 0s 2019-12-09T09:27:15.078 INFO:teuthology.orchestra.run.smithi035.stderr: 36900K .......... .......... .......... .......... .......... 50% 223M 0s 2019-12-09T09:27:15.078 INFO:teuthology.orchestra.run.smithi035.stderr: 36950K .......... .......... .......... .......... .......... 50% 184M 0s 2019-12-09T09:27:15.078 INFO:teuthology.orchestra.run.smithi035.stderr: 37000K .......... .......... .......... .......... .......... 50% 222M 0s 2019-12-09T09:27:15.079 INFO:teuthology.orchestra.run.smithi035.stderr: 37050K .......... .......... .......... .......... .......... 50% 250M 0s 2019-12-09T09:27:15.079 INFO:teuthology.orchestra.run.smithi035.stderr: 37100K .......... .......... .......... .......... .......... 50% 213M 0s 2019-12-09T09:27:15.079 INFO:teuthology.orchestra.run.smithi035.stderr: 37150K .......... .......... .......... .......... .......... 50% 187M 0s 2019-12-09T09:27:15.079 INFO:teuthology.orchestra.run.smithi035.stderr: 37200K .......... .......... .......... .......... .......... 50% 255M 0s 2019-12-09T09:27:15.079 INFO:teuthology.orchestra.run.smithi035.stderr: 37250K .......... .......... .......... .......... .......... 50% 229M 0s 2019-12-09T09:27:15.079 INFO:teuthology.orchestra.run.smithi035.stderr: 37300K .......... .......... .......... .......... .......... 50% 241M 0s 2019-12-09T09:27:15.079 INFO:teuthology.orchestra.run.smithi035.stderr: 37350K .......... .......... .......... .......... .......... 50% 200M 0s 2019-12-09T09:27:15.079 INFO:teuthology.orchestra.run.smithi035.stderr: 37400K .......... .......... .......... .......... .......... 50% 232M 0s 2019-12-09T09:27:15.080 INFO:teuthology.orchestra.run.smithi035.stderr: 37450K .......... .......... .......... .......... .......... 50% 235M 0s 2019-12-09T09:27:15.080 INFO:teuthology.orchestra.run.smithi035.stderr: 37500K .......... .......... .......... .......... .......... 51% 243M 0s 2019-12-09T09:27:15.080 INFO:teuthology.orchestra.run.smithi035.stderr: 37550K .......... .......... .......... .......... .......... 51% 193M 0s 2019-12-09T09:27:15.080 INFO:teuthology.orchestra.run.smithi035.stderr: 37600K .......... .......... .......... .......... .......... 51% 212M 0s 2019-12-09T09:27:15.080 INFO:teuthology.orchestra.run.smithi035.stderr: 37650K .......... .......... .......... .......... .......... 51% 222M 0s 2019-12-09T09:27:15.080 INFO:teuthology.orchestra.run.smithi035.stderr: 37700K .......... .......... .......... .......... .......... 51% 219M 0s 2019-12-09T09:27:15.080 INFO:teuthology.orchestra.run.smithi035.stderr: 37750K .......... .......... .......... .......... .......... 51% 218M 0s 2019-12-09T09:27:15.081 INFO:teuthology.orchestra.run.smithi035.stderr: 37800K .......... .......... .......... .......... .......... 51% 229M 0s 2019-12-09T09:27:15.081 INFO:teuthology.orchestra.run.smithi035.stderr: 37850K .......... .......... .......... .......... .......... 51% 213M 0s 2019-12-09T09:27:15.081 INFO:teuthology.orchestra.run.smithi035.stderr: 37900K .......... .......... .......... .......... .......... 51% 223M 0s 2019-12-09T09:27:15.081 INFO:teuthology.orchestra.run.smithi035.stderr: 37950K .......... .......... .......... .......... .......... 51% 208M 0s 2019-12-09T09:27:15.081 INFO:teuthology.orchestra.run.smithi035.stderr: 38000K .......... .......... .......... .......... .......... 51% 229M 0s 2019-12-09T09:27:15.081 INFO:teuthology.orchestra.run.smithi035.stderr: 38050K .......... .......... .......... .......... .......... 51% 252M 0s 2019-12-09T09:27:15.081 INFO:teuthology.orchestra.run.smithi035.stderr: 38100K .......... .......... .......... .......... .......... 51% 344M 0s 2019-12-09T09:27:15.082 INFO:teuthology.orchestra.run.smithi035.stderr: 38150K .......... .......... .......... .......... .......... 51% 277M 0s 2019-12-09T09:27:15.082 INFO:teuthology.orchestra.run.smithi035.stderr: 38200K .......... .......... .......... .......... .......... 51% 306M 0s 2019-12-09T09:27:15.082 INFO:teuthology.orchestra.run.smithi035.stderr: 38250K .......... .......... .......... .......... .......... 52% 255M 0s 2019-12-09T09:27:15.082 INFO:teuthology.orchestra.run.smithi035.stderr: 38300K .......... .......... .......... .......... .......... 52% 342M 0s 2019-12-09T09:27:15.082 INFO:teuthology.orchestra.run.smithi035.stderr: 38350K .......... .......... .......... .......... .......... 52% 237M 0s 2019-12-09T09:27:15.082 INFO:teuthology.orchestra.run.smithi035.stderr: 38400K .......... .......... .......... .......... .......... 52% 293M 0s 2019-12-09T09:27:15.082 INFO:teuthology.orchestra.run.smithi035.stderr: 38450K .......... .......... .......... .......... .......... 52% 317M 0s 2019-12-09T09:27:15.082 INFO:teuthology.orchestra.run.smithi035.stderr: 38500K .......... .......... .......... .......... .......... 52% 314M 0s 2019-12-09T09:27:15.083 INFO:teuthology.orchestra.run.smithi035.stderr: 38550K .......... .......... .......... .......... .......... 52% 271M 0s 2019-12-09T09:27:15.083 INFO:teuthology.orchestra.run.smithi035.stderr: 38600K .......... .......... .......... .......... .......... 52% 310M 0s 2019-12-09T09:27:15.083 INFO:teuthology.orchestra.run.smithi035.stderr: 38650K .......... .......... .......... .......... .......... 52% 305M 0s 2019-12-09T09:27:15.083 INFO:teuthology.orchestra.run.smithi035.stderr: 38700K .......... .......... .......... .......... .......... 52% 351M 0s 2019-12-09T09:27:15.083 INFO:teuthology.orchestra.run.smithi035.stderr: 38750K .......... .......... .......... .......... .......... 52% 265M 0s 2019-12-09T09:27:15.083 INFO:teuthology.orchestra.run.smithi035.stderr: 38800K .......... .......... .......... .......... .......... 52% 314M 0s 2019-12-09T09:27:15.083 INFO:teuthology.orchestra.run.smithi035.stderr: 38850K .......... .......... .......... .......... .......... 52% 310M 0s 2019-12-09T09:27:15.084 INFO:teuthology.orchestra.run.smithi035.stderr: 38900K .......... .......... .......... .......... .......... 52% 317M 0s 2019-12-09T09:27:15.084 INFO:teuthology.orchestra.run.smithi035.stderr: 38950K .......... .......... .......... .......... .......... 53% 272M 0s 2019-12-09T09:27:15.084 INFO:teuthology.orchestra.run.smithi035.stderr: 39000K .......... .......... .......... .......... .......... 53% 314M 0s 2019-12-09T09:27:15.084 INFO:teuthology.orchestra.run.smithi035.stderr: 39050K .......... .......... .......... .......... .......... 53% 327M 0s 2019-12-09T09:27:15.084 INFO:teuthology.orchestra.run.smithi035.stderr: 39100K .......... .......... .......... .......... .......... 53% 347M 0s 2019-12-09T09:27:15.084 INFO:teuthology.orchestra.run.smithi035.stderr: 39150K .......... .......... .......... .......... .......... 53% 283M 0s 2019-12-09T09:27:15.084 INFO:teuthology.orchestra.run.smithi035.stderr: 39200K .......... .......... .......... .......... .......... 53% 343M 0s 2019-12-09T09:27:15.085 INFO:teuthology.orchestra.run.smithi035.stderr: 39250K .......... .......... .......... .......... .......... 53% 318M 0s 2019-12-09T09:27:15.085 INFO:teuthology.orchestra.run.smithi035.stderr: 39300K .......... .......... .......... .......... .......... 53% 335M 0s 2019-12-09T09:27:15.085 INFO:teuthology.orchestra.run.smithi035.stderr: 39350K .......... .......... .......... .......... .......... 53% 270M 0s 2019-12-09T09:27:15.085 INFO:teuthology.orchestra.run.smithi035.stderr: 39400K .......... .......... .......... .......... .......... 53% 261M 0s 2019-12-09T09:27:15.085 INFO:teuthology.orchestra.run.smithi035.stderr: 39450K .......... .......... .......... .......... .......... 53% 307M 0s 2019-12-09T09:27:15.085 INFO:teuthology.orchestra.run.smithi035.stderr: 39500K .......... .......... .......... .......... .......... 53% 309M 0s 2019-12-09T09:27:15.085 INFO:teuthology.orchestra.run.smithi035.stderr: 39550K .......... .......... .......... .......... .......... 53% 245M 0s 2019-12-09T09:27:15.086 INFO:teuthology.orchestra.run.smithi035.stderr: 39600K .......... .......... .......... .......... .......... 53% 346M 0s 2019-12-09T09:27:15.086 INFO:teuthology.orchestra.run.smithi035.stderr: 39650K .......... .......... .......... .......... .......... 53% 306M 0s 2019-12-09T09:27:15.086 INFO:teuthology.orchestra.run.smithi035.stderr: 39700K .......... .......... .......... .......... .......... 54% 299M 0s 2019-12-09T09:27:15.086 INFO:teuthology.orchestra.run.smithi035.stderr: 39750K .......... .......... .......... .......... .......... 54% 233M 0s 2019-12-09T09:27:15.086 INFO:teuthology.orchestra.run.smithi035.stderr: 39800K .......... .......... .......... .......... .......... 54% 248M 0s 2019-12-09T09:27:15.086 INFO:teuthology.orchestra.run.smithi035.stderr: 39850K .......... .......... .......... .......... .......... 54% 297M 0s 2019-12-09T09:27:15.086 INFO:teuthology.orchestra.run.smithi035.stderr: 39900K .......... .......... .......... .......... .......... 54% 320M 0s 2019-12-09T09:27:15.087 INFO:teuthology.orchestra.run.smithi035.stderr: 39950K .......... .......... .......... .......... .......... 54% 311M 0s 2019-12-09T09:27:15.087 INFO:teuthology.orchestra.run.smithi035.stderr: 40000K .......... .......... .......... .......... .......... 54% 280M 0s 2019-12-09T09:27:15.087 INFO:teuthology.orchestra.run.smithi035.stderr: 40050K .......... .......... .......... .......... .......... 54% 288M 0s 2019-12-09T09:27:15.087 INFO:teuthology.orchestra.run.smithi035.stderr: 40100K .......... .......... .......... .......... .......... 54% 311M 0s 2019-12-09T09:27:15.087 INFO:teuthology.orchestra.run.smithi035.stderr: 40150K .......... .......... .......... .......... .......... 54% 256M 0s 2019-12-09T09:27:15.087 INFO:teuthology.orchestra.run.smithi035.stderr: 40200K .......... .......... .......... .......... .......... 54% 325M 0s 2019-12-09T09:27:15.087 INFO:teuthology.orchestra.run.smithi035.stderr: 40250K .......... .......... .......... .......... .......... 54% 318M 0s 2019-12-09T09:27:15.087 INFO:teuthology.orchestra.run.smithi035.stderr: 40300K .......... .......... .......... .......... .......... 54% 291M 0s 2019-12-09T09:27:15.088 INFO:teuthology.orchestra.run.smithi035.stderr: 40350K .......... .......... .......... .......... .......... 54% 298M 0s 2019-12-09T09:27:15.088 INFO:teuthology.orchestra.run.smithi035.stderr: 40400K .......... .......... .......... .......... .......... 54% 363M 0s 2019-12-09T09:27:15.088 INFO:teuthology.orchestra.run.smithi035.stderr: 40450K .......... .......... .......... .......... .......... 55% 366M 0s 2019-12-09T09:27:15.088 INFO:teuthology.orchestra.run.smithi035.stderr: 40500K .......... .......... .......... .......... .......... 55% 353M 0s 2019-12-09T09:27:15.088 INFO:teuthology.orchestra.run.smithi035.stderr: 40550K .......... .......... .......... .......... .......... 55% 251M 0s 2019-12-09T09:27:15.088 INFO:teuthology.orchestra.run.smithi035.stderr: 40600K .......... .......... .......... .......... .......... 55% 349M 0s 2019-12-09T09:27:15.088 INFO:teuthology.orchestra.run.smithi035.stderr: 40650K .......... .......... .......... .......... .......... 55% 312M 0s 2019-12-09T09:27:15.089 INFO:teuthology.orchestra.run.smithi035.stderr: 40700K .......... .......... .......... .......... .......... 55% 304M 0s 2019-12-09T09:27:15.089 INFO:teuthology.orchestra.run.smithi035.stderr: 40750K .......... .......... .......... .......... .......... 55% 278M 0s 2019-12-09T09:27:15.089 INFO:teuthology.orchestra.run.smithi035.stderr: 40800K .......... .......... .......... .......... .......... 55% 340M 0s 2019-12-09T09:27:15.089 INFO:teuthology.orchestra.run.smithi035.stderr: 40850K .......... .......... .......... .......... .......... 55% 308M 0s 2019-12-09T09:27:15.089 INFO:teuthology.orchestra.run.smithi035.stderr: 40900K .......... .......... .......... .......... .......... 55% 341M 0s 2019-12-09T09:27:15.089 INFO:teuthology.orchestra.run.smithi035.stderr: 40950K .......... .......... .......... .......... .......... 55% 289M 0s 2019-12-09T09:27:15.089 INFO:teuthology.orchestra.run.smithi035.stderr: 41000K .......... .......... .......... .......... .......... 55% 340M 0s 2019-12-09T09:27:15.090 INFO:teuthology.orchestra.run.smithi035.stderr: 41050K .......... .......... .......... .......... .......... 55% 353M 0s 2019-12-09T09:27:15.090 INFO:teuthology.orchestra.run.smithi035.stderr: 41100K .......... .......... .......... .......... .......... 55% 350M 0s 2019-12-09T09:27:15.090 INFO:teuthology.orchestra.run.smithi035.stderr: 41150K .......... .......... .......... .......... .......... 56% 289M 0s 2019-12-09T09:27:15.090 INFO:teuthology.orchestra.run.smithi035.stderr: 41200K .......... .......... .......... .......... .......... 56% 343M 0s 2019-12-09T09:27:15.090 INFO:teuthology.orchestra.run.smithi035.stderr: 41250K .......... .......... .......... .......... .......... 56% 350M 0s 2019-12-09T09:27:15.090 INFO:teuthology.orchestra.run.smithi035.stderr: 41300K .......... .......... .......... .......... .......... 56% 331M 0s 2019-12-09T09:27:15.090 INFO:teuthology.orchestra.run.smithi035.stderr: 41350K .......... .......... .......... .......... .......... 56% 293M 0s 2019-12-09T09:27:15.091 INFO:teuthology.orchestra.run.smithi035.stderr: 41400K .......... .......... .......... .......... .......... 56% 338M 0s 2019-12-09T09:27:15.091 INFO:teuthology.orchestra.run.smithi035.stderr: 41450K .......... .......... .......... .......... .......... 56% 322M 0s 2019-12-09T09:27:15.091 INFO:teuthology.orchestra.run.smithi035.stderr: 41500K .......... .......... .......... .......... .......... 56% 352M 0s 2019-12-09T09:27:15.091 INFO:teuthology.orchestra.run.smithi035.stderr: 41550K .......... .......... .......... .......... .......... 56% 308M 0s 2019-12-09T09:27:15.091 INFO:teuthology.orchestra.run.smithi035.stderr: 41600K .......... .......... .......... .......... .......... 56% 326M 0s 2019-12-09T09:27:15.091 INFO:teuthology.orchestra.run.smithi035.stderr: 41650K .......... .......... .......... .......... .......... 56% 341M 0s 2019-12-09T09:27:15.091 INFO:teuthology.orchestra.run.smithi035.stderr: 41700K .......... .......... .......... .......... .......... 56% 364M 0s 2019-12-09T09:27:15.092 INFO:teuthology.orchestra.run.smithi035.stderr: 41750K .......... .......... .......... .......... .......... 56% 296M 0s 2019-12-09T09:27:15.092 INFO:teuthology.orchestra.run.smithi035.stderr: 41800K .......... .......... .......... .......... .......... 56% 305M 0s 2019-12-09T09:27:15.092 INFO:teuthology.orchestra.run.smithi035.stderr: 41850K .......... .......... .......... .......... .......... 56% 326M 0s 2019-12-09T09:27:15.092 INFO:teuthology.orchestra.run.smithi035.stderr: 41900K .......... .......... .......... .......... .......... 57% 262M 0s 2019-12-09T09:27:15.092 INFO:teuthology.orchestra.run.smithi035.stderr: 41950K .......... .......... .......... .......... .......... 57% 237M 0s 2019-12-09T09:27:15.092 INFO:teuthology.orchestra.run.smithi035.stderr: 42000K .......... .......... .......... .......... .......... 57% 313M 0s 2019-12-09T09:27:15.093 INFO:teuthology.orchestra.run.smithi035.stderr: 42050K .......... .......... .......... .......... .......... 57% 334M 0s 2019-12-09T09:27:15.093 INFO:teuthology.orchestra.run.smithi035.stderr: 42100K .......... .......... .......... .......... .......... 57% 307M 0s 2019-12-09T09:27:15.093 INFO:teuthology.orchestra.run.smithi035.stderr: 42150K .......... .......... .......... .......... .......... 57% 249M 0s 2019-12-09T09:27:15.093 INFO:teuthology.orchestra.run.smithi035.stderr: 42200K .......... .......... .......... .......... .......... 57% 279M 0s 2019-12-09T09:27:15.093 INFO:teuthology.orchestra.run.smithi035.stderr: 42250K .......... .......... .......... .......... .......... 57% 324M 0s 2019-12-09T09:27:15.093 INFO:teuthology.orchestra.run.smithi035.stderr: 42300K .......... .......... .......... .......... .......... 57% 305M 0s 2019-12-09T09:27:15.093 INFO:teuthology.orchestra.run.smithi035.stderr: 42350K .......... .......... .......... .......... .......... 57% 275M 0s 2019-12-09T09:27:15.094 INFO:teuthology.orchestra.run.smithi035.stderr: 42400K .......... .......... .......... .......... .......... 57% 303M 0s 2019-12-09T09:27:15.094 INFO:teuthology.orchestra.run.smithi035.stderr: 42450K .......... .......... .......... .......... .......... 57% 283M 0s 2019-12-09T09:27:15.094 INFO:teuthology.orchestra.run.smithi035.stderr: 42500K .......... .......... .......... .......... .......... 57% 302M 0s 2019-12-09T09:27:15.094 INFO:teuthology.orchestra.run.smithi035.stderr: 42550K .......... .......... .......... .......... .......... 57% 234M 0s 2019-12-09T09:27:15.094 INFO:teuthology.orchestra.run.smithi035.stderr: 42600K .......... .......... .......... .......... .......... 57% 326M 0s 2019-12-09T09:27:15.094 INFO:teuthology.orchestra.run.smithi035.stderr: 42650K .......... .......... .......... .......... .......... 58% 319M 0s 2019-12-09T09:27:15.094 INFO:teuthology.orchestra.run.smithi035.stderr: 42700K .......... .......... .......... .......... .......... 58% 323M 0s 2019-12-09T09:27:15.094 INFO:teuthology.orchestra.run.smithi035.stderr: 42750K .......... .......... .......... .......... .......... 58% 317M 0s 2019-12-09T09:27:15.095 INFO:teuthology.orchestra.run.smithi035.stderr: 42800K .......... .......... .......... .......... .......... 58% 350M 0s 2019-12-09T09:27:15.095 INFO:teuthology.orchestra.run.smithi035.stderr: 42850K .......... .......... .......... .......... .......... 58% 338M 0s 2019-12-09T09:27:15.095 INFO:teuthology.orchestra.run.smithi035.stderr: 42900K .......... .......... .......... .......... .......... 58% 370M 0s 2019-12-09T09:27:15.095 INFO:teuthology.orchestra.run.smithi035.stderr: 42950K .......... .......... .......... .......... .......... 58% 301M 0s 2019-12-09T09:27:15.095 INFO:teuthology.orchestra.run.smithi035.stderr: 43000K .......... .......... .......... .......... .......... 58% 342M 0s 2019-12-09T09:27:15.095 INFO:teuthology.orchestra.run.smithi035.stderr: 43050K .......... .......... .......... .......... .......... 58% 388M 0s 2019-12-09T09:27:15.095 INFO:teuthology.orchestra.run.smithi035.stderr: 43100K .......... .......... .......... .......... .......... 58% 363M 0s 2019-12-09T09:27:15.096 INFO:teuthology.orchestra.run.smithi035.stderr: 43150K .......... .......... .......... .......... .......... 58% 306M 0s 2019-12-09T09:27:15.096 INFO:teuthology.orchestra.run.smithi035.stderr: 43200K .......... .......... .......... .......... .......... 58% 315M 0s 2019-12-09T09:27:15.096 INFO:teuthology.orchestra.run.smithi035.stderr: 43250K .......... .......... .......... .......... .......... 58% 343M 0s 2019-12-09T09:27:15.096 INFO:teuthology.orchestra.run.smithi035.stderr: 43300K .......... .......... .......... .......... .......... 58% 363M 0s 2019-12-09T09:27:15.096 INFO:teuthology.orchestra.run.smithi035.stderr: 43350K .......... .......... .......... .......... .......... 59% 324M 0s 2019-12-09T09:27:15.096 INFO:teuthology.orchestra.run.smithi035.stderr: 43400K .......... .......... .......... .......... .......... 59% 339M 0s 2019-12-09T09:27:15.096 INFO:teuthology.orchestra.run.smithi035.stderr: 43450K .......... .......... .......... .......... .......... 59% 316M 0s 2019-12-09T09:27:15.096 INFO:teuthology.orchestra.run.smithi035.stderr: 43500K .......... .......... .......... .......... .......... 59% 343M 0s 2019-12-09T09:27:15.097 INFO:teuthology.orchestra.run.smithi035.stderr: 43550K .......... .......... .......... .......... .......... 59% 291M 0s 2019-12-09T09:27:15.097 INFO:teuthology.orchestra.run.smithi035.stderr: 43600K .......... .......... .......... .......... .......... 59% 324M 0s 2019-12-09T09:27:15.097 INFO:teuthology.orchestra.run.smithi035.stderr: 43650K .......... .......... .......... .......... .......... 59% 368M 0s 2019-12-09T09:27:15.097 INFO:teuthology.orchestra.run.smithi035.stderr: 43700K .......... .......... .......... .......... .......... 59% 329M 0s 2019-12-09T09:27:15.097 INFO:teuthology.orchestra.run.smithi035.stderr: 43750K .......... .......... .......... .......... .......... 59% 325M 0s 2019-12-09T09:27:15.097 INFO:teuthology.orchestra.run.smithi035.stderr: 43800K .......... .......... .......... .......... .......... 59% 352M 0s 2019-12-09T09:27:15.098 INFO:teuthology.orchestra.run.smithi035.stderr: 43850K .......... .......... .......... .......... .......... 59% 321M 0s 2019-12-09T09:27:15.098 INFO:teuthology.orchestra.run.smithi035.stderr: 43900K .......... .......... .......... .......... .......... 59% 374M 0s 2019-12-09T09:27:15.098 INFO:teuthology.orchestra.run.smithi035.stderr: 43950K .......... .......... .......... .......... .......... 59% 272M 0s 2019-12-09T09:27:15.098 INFO:teuthology.orchestra.run.smithi035.stderr: 44000K .......... .......... .......... .......... .......... 59% 347M 0s 2019-12-09T09:27:15.098 INFO:teuthology.orchestra.run.smithi035.stderr: 44050K .......... .......... .......... .......... .......... 59% 343M 0s 2019-12-09T09:27:15.098 INFO:teuthology.orchestra.run.smithi035.stderr: 44100K .......... .......... .......... .......... .......... 60% 353M 0s 2019-12-09T09:27:15.099 INFO:teuthology.orchestra.run.smithi035.stderr: 44150K .......... .......... .......... .......... .......... 60% 325M 0s 2019-12-09T09:27:15.099 INFO:teuthology.orchestra.run.smithi035.stderr: 44200K .......... .......... .......... .......... .......... 60% 337M 0s 2019-12-09T09:27:15.099 INFO:teuthology.orchestra.run.smithi035.stderr: 44250K .......... .......... .......... .......... .......... 60% 383M 0s 2019-12-09T09:27:15.099 INFO:teuthology.orchestra.run.smithi035.stderr: 44300K .......... .......... .......... .......... .......... 60% 342M 0s 2019-12-09T09:27:15.099 INFO:teuthology.orchestra.run.smithi035.stderr: 44350K .......... .......... .......... .......... .......... 60% 308M 0s 2019-12-09T09:27:15.099 INFO:teuthology.orchestra.run.smithi035.stderr: 44400K .......... .......... .......... .......... .......... 60% 339M 0s 2019-12-09T09:27:15.099 INFO:teuthology.orchestra.run.smithi035.stderr: 44450K .......... .......... .......... .......... .......... 60% 351M 0s 2019-12-09T09:27:15.100 INFO:teuthology.orchestra.run.smithi035.stderr: 44500K .......... .......... .......... .......... .......... 60% 354M 0s 2019-12-09T09:27:15.100 INFO:teuthology.orchestra.run.smithi035.stderr: 44550K .......... .......... .......... .......... .......... 60% 291M 0s 2019-12-09T09:27:15.100 INFO:teuthology.orchestra.run.smithi035.stderr: 44600K .......... .......... .......... .......... .......... 60% 352M 0s 2019-12-09T09:27:15.100 INFO:teuthology.orchestra.run.smithi035.stderr: 44650K .......... .......... .......... .......... .......... 60% 338M 0s 2019-12-09T09:27:15.100 INFO:teuthology.orchestra.run.smithi035.stderr: 44700K .......... .......... .......... .......... .......... 60% 375M 0s 2019-12-09T09:27:15.100 INFO:teuthology.orchestra.run.smithi035.stderr: 44750K .......... .......... .......... .......... .......... 60% 282M 0s 2019-12-09T09:27:15.101 INFO:teuthology.orchestra.run.smithi035.stderr: 44800K .......... .......... .......... .......... .......... 60% 375M 0s 2019-12-09T09:27:15.101 INFO:teuthology.orchestra.run.smithi035.stderr: 44850K .......... .......... .......... .......... .......... 61% 387M 0s 2019-12-09T09:27:15.101 INFO:teuthology.orchestra.run.smithi035.stderr: 44900K .......... .......... .......... .......... .......... 61% 364M 0s 2019-12-09T09:27:15.101 INFO:teuthology.orchestra.run.smithi035.stderr: 44950K .......... .......... .......... .......... .......... 61% 326M 0s 2019-12-09T09:27:15.101 INFO:teuthology.orchestra.run.smithi035.stderr: 45000K .......... .......... .......... .......... .......... 61% 380M 0s 2019-12-09T09:27:15.101 INFO:teuthology.orchestra.run.smithi035.stderr: 45050K .......... .......... .......... .......... .......... 61% 383M 0s 2019-12-09T09:27:15.101 INFO:teuthology.orchestra.run.smithi035.stderr: 45100K .......... .......... .......... .......... .......... 61% 392M 0s 2019-12-09T09:27:15.102 INFO:teuthology.orchestra.run.smithi035.stderr: 45150K .......... .......... .......... .......... .......... 61% 324M 0s 2019-12-09T09:27:15.103 INFO:teuthology.orchestra.run.smithi035.stderr: 45200K .......... .......... .......... .......... .......... 61% 387M 0s 2019-12-09T09:27:15.103 INFO:teuthology.orchestra.run.smithi035.stderr: 45250K .......... .......... .......... .......... .......... 61% 246M 0s 2019-12-09T09:27:15.103 INFO:teuthology.orchestra.run.smithi035.stderr: 45300K .......... .......... .......... .......... .......... 61% 268M 0s 2019-12-09T09:27:15.103 INFO:teuthology.orchestra.run.smithi035.stderr: 45350K .......... .......... .......... .......... .......... 61% 276M 0s 2019-12-09T09:27:15.104 INFO:teuthology.orchestra.run.smithi035.stderr: 45400K .......... .......... .......... .......... .......... 61% 325M 0s 2019-12-09T09:27:15.104 INFO:teuthology.orchestra.run.smithi035.stderr: 45450K .......... .......... .......... .......... .......... 61% 244M 0s 2019-12-09T09:27:15.104 INFO:teuthology.orchestra.run.smithi035.stderr: 45500K .......... .......... .......... .......... .......... 61% 225M 0s 2019-12-09T09:27:15.104 INFO:teuthology.orchestra.run.smithi035.stderr: 45550K .......... .......... .......... .......... .......... 61% 219M 0s 2019-12-09T09:27:15.104 INFO:teuthology.orchestra.run.smithi035.stderr: 45600K .......... .......... .......... .......... .......... 62% 280M 0s 2019-12-09T09:27:15.104 INFO:teuthology.orchestra.run.smithi035.stderr: 45650K .......... .......... .......... .......... .......... 62% 325M 0s 2019-12-09T09:27:15.104 INFO:teuthology.orchestra.run.smithi035.stderr: 45700K .......... .......... .......... .......... .......... 62% 348M 0s 2019-12-09T09:27:15.105 INFO:teuthology.orchestra.run.smithi035.stderr: 45750K .......... .......... .......... .......... .......... 62% 243M 0s 2019-12-09T09:27:15.105 INFO:teuthology.orchestra.run.smithi035.stderr: 45800K .......... .......... .......... .......... .......... 62% 312M 0s 2019-12-09T09:27:15.105 INFO:teuthology.orchestra.run.smithi035.stderr: 45850K .......... .......... .......... .......... .......... 62% 333M 0s 2019-12-09T09:27:15.105 INFO:teuthology.orchestra.run.smithi035.stderr: 45900K .......... .......... .......... .......... .......... 62% 330M 0s 2019-12-09T09:27:15.105 INFO:teuthology.orchestra.run.smithi035.stderr: 45950K .......... .......... .......... .......... .......... 62% 228M 0s 2019-12-09T09:27:15.105 INFO:teuthology.orchestra.run.smithi035.stderr: 46000K .......... .......... .......... .......... .......... 62% 276M 0s 2019-12-09T09:27:15.105 INFO:teuthology.orchestra.run.smithi035.stderr: 46050K .......... .......... .......... .......... .......... 62% 256M 0s 2019-12-09T09:27:15.106 INFO:teuthology.orchestra.run.smithi035.stderr: 46100K .......... .......... .......... .......... .......... 62% 316M 0s 2019-12-09T09:27:15.106 INFO:teuthology.orchestra.run.smithi035.stderr: 46150K .......... .......... .......... .......... .......... 62% 272M 0s 2019-12-09T09:27:15.106 INFO:teuthology.orchestra.run.smithi035.stderr: 46200K .......... .......... .......... .......... .......... 62% 286M 0s 2019-12-09T09:27:15.106 INFO:teuthology.orchestra.run.smithi035.stderr: 46250K .......... .......... .......... .......... .......... 62% 253M 0s 2019-12-09T09:27:15.106 INFO:teuthology.orchestra.run.smithi035.stderr: 46300K .......... .......... .......... .......... .......... 63% 274M 0s 2019-12-09T09:27:15.106 INFO:teuthology.orchestra.run.smithi035.stderr: 46350K .......... .......... .......... .......... .......... 63% 299M 0s 2019-12-09T09:27:15.106 INFO:teuthology.orchestra.run.smithi035.stderr: 46400K .......... .......... .......... .......... .......... 63% 318M 0s 2019-12-09T09:27:15.106 INFO:teuthology.orchestra.run.smithi035.stderr: 46450K .......... .......... .......... .......... .......... 63% 266M 0s 2019-12-09T09:27:15.107 INFO:teuthology.orchestra.run.smithi035.stderr: 46500K .......... .......... .......... .......... .......... 63% 262M 0s 2019-12-09T09:27:15.107 INFO:teuthology.orchestra.run.smithi035.stderr: 46550K .......... .......... .......... .......... .......... 63% 299M 0s 2019-12-09T09:27:15.107 INFO:teuthology.orchestra.run.smithi035.stderr: 46600K .......... .......... .......... .......... .......... 63% 275M 0s 2019-12-09T09:27:15.107 INFO:teuthology.orchestra.run.smithi035.stderr: 46650K .......... .......... .......... .......... .......... 63% 299M 0s 2019-12-09T09:27:15.107 INFO:teuthology.orchestra.run.smithi035.stderr: 46700K .......... .......... .......... .......... .......... 63% 237M 0s 2019-12-09T09:27:15.107 INFO:teuthology.orchestra.run.smithi035.stderr: 46750K .......... .......... .......... .......... .......... 63% 262M 0s 2019-12-09T09:27:15.107 INFO:teuthology.orchestra.run.smithi035.stderr: 46800K .......... .......... .......... .......... .......... 63% 316M 0s 2019-12-09T09:27:15.108 INFO:teuthology.orchestra.run.smithi035.stderr: 46850K .......... .......... .......... .......... .......... 63% 322M 0s 2019-12-09T09:27:15.108 INFO:teuthology.orchestra.run.smithi035.stderr: 46900K .......... .......... .......... .......... .......... 63% 286M 0s 2019-12-09T09:27:15.108 INFO:teuthology.orchestra.run.smithi035.stderr: 46950K .......... .......... .......... .......... .......... 63% 207M 0s 2019-12-09T09:27:15.108 INFO:teuthology.orchestra.run.smithi035.stderr: 47000K .......... .......... .......... .......... .......... 63% 291M 0s 2019-12-09T09:27:15.108 INFO:teuthology.orchestra.run.smithi035.stderr: 47050K .......... .......... .......... .......... .......... 64% 220M 0s 2019-12-09T09:27:15.108 INFO:teuthology.orchestra.run.smithi035.stderr: 47100K .......... .......... .......... .......... .......... 64% 331M 0s 2019-12-09T09:27:15.108 INFO:teuthology.orchestra.run.smithi035.stderr: 47150K .......... .......... .......... .......... .......... 64% 283M 0s 2019-12-09T09:27:15.109 INFO:teuthology.orchestra.run.smithi035.stderr: 47200K .......... .......... .......... .......... .......... 64% 320M 0s 2019-12-09T09:27:15.109 INFO:teuthology.orchestra.run.smithi035.stderr: 47250K .......... .......... .......... .......... .......... 64% 305M 0s 2019-12-09T09:27:15.109 INFO:teuthology.orchestra.run.smithi035.stderr: 47300K .......... .......... .......... .......... .......... 64% 325M 0s 2019-12-09T09:27:15.109 INFO:teuthology.orchestra.run.smithi035.stderr: 47350K .......... .......... .......... .......... .......... 64% 268M 0s 2019-12-09T09:27:15.109 INFO:teuthology.orchestra.run.smithi035.stderr: 47400K .......... .......... .......... .......... .......... 64% 261M 0s 2019-12-09T09:27:15.109 INFO:teuthology.orchestra.run.smithi035.stderr: 47450K .......... .......... .......... .......... .......... 64% 218M 0s 2019-12-09T09:27:15.109 INFO:teuthology.orchestra.run.smithi035.stderr: 47500K .......... .......... .......... .......... .......... 64% 337M 0s 2019-12-09T09:27:15.110 INFO:teuthology.orchestra.run.smithi035.stderr: 47550K .......... .......... .......... .......... .......... 64% 264M 0s 2019-12-09T09:27:15.110 INFO:teuthology.orchestra.run.smithi035.stderr: 47600K .......... .......... .......... .......... .......... 64% 285M 0s 2019-12-09T09:27:15.110 INFO:teuthology.orchestra.run.smithi035.stderr: 47650K .......... .......... .......... .......... .......... 64% 278M 0s 2019-12-09T09:27:15.110 INFO:teuthology.orchestra.run.smithi035.stderr: 47700K .......... .......... .......... .......... .......... 64% 316M 0s 2019-12-09T09:27:15.110 INFO:teuthology.orchestra.run.smithi035.stderr: 47750K .......... .......... .......... .......... .......... 64% 204M 0s 2019-12-09T09:27:15.110 INFO:teuthology.orchestra.run.smithi035.stderr: 47800K .......... .......... .......... .......... .......... 65% 314M 0s 2019-12-09T09:27:15.110 INFO:teuthology.orchestra.run.smithi035.stderr: 47850K .......... .......... .......... .......... .......... 65% 353M 0s 2019-12-09T09:27:15.111 INFO:teuthology.orchestra.run.smithi035.stderr: 47900K .......... .......... .......... .......... .......... 65% 307M 0s 2019-12-09T09:27:15.111 INFO:teuthology.orchestra.run.smithi035.stderr: 47950K .......... .......... .......... .......... .......... 65% 208M 0s 2019-12-09T09:27:15.111 INFO:teuthology.orchestra.run.smithi035.stderr: 48000K .......... .......... .......... .......... .......... 65% 300M 0s 2019-12-09T09:27:15.111 INFO:teuthology.orchestra.run.smithi035.stderr: 48050K .......... .......... .......... .......... .......... 65% 302M 0s 2019-12-09T09:27:15.111 INFO:teuthology.orchestra.run.smithi035.stderr: 48100K .......... .......... .......... .......... .......... 65% 269M 0s 2019-12-09T09:27:15.111 INFO:teuthology.orchestra.run.smithi035.stderr: 48150K .......... .......... .......... .......... .......... 65% 235M 0s 2019-12-09T09:27:15.112 INFO:teuthology.orchestra.run.smithi035.stderr: 48200K .......... .......... .......... .......... .......... 65% 312M 0s 2019-12-09T09:27:15.112 INFO:teuthology.orchestra.run.smithi035.stderr: 48250K .......... .......... .......... .......... .......... 65% 283M 0s 2019-12-09T09:27:15.112 INFO:teuthology.orchestra.run.smithi035.stderr: 48300K .......... .......... .......... .......... .......... 65% 309M 0s 2019-12-09T09:27:15.112 INFO:teuthology.orchestra.run.smithi035.stderr: 48350K .......... .......... .......... .......... .......... 65% 286M 0s 2019-12-09T09:27:15.112 INFO:teuthology.orchestra.run.smithi035.stderr: 48400K .......... .......... .......... .......... .......... 65% 263M 0s 2019-12-09T09:27:15.112 INFO:teuthology.orchestra.run.smithi035.stderr: 48450K .......... .......... .......... .......... .......... 65% 313M 0s 2019-12-09T09:27:15.112 INFO:teuthology.orchestra.run.smithi035.stderr: 48500K .......... .......... .......... .......... .......... 66% 253M 0s 2019-12-09T09:27:15.113 INFO:teuthology.orchestra.run.smithi035.stderr: 48550K .......... .......... .......... .......... .......... 66% 284M 0s 2019-12-09T09:27:15.113 INFO:teuthology.orchestra.run.smithi035.stderr: 48600K .......... .......... .......... .......... .......... 66% 271M 0s 2019-12-09T09:27:15.113 INFO:teuthology.orchestra.run.smithi035.stderr: 48650K .......... .......... .......... .......... .......... 66% 313M 0s 2019-12-09T09:27:15.113 INFO:teuthology.orchestra.run.smithi035.stderr: 48700K .......... .......... .......... .......... .......... 66% 284M 0s 2019-12-09T09:27:15.113 INFO:teuthology.orchestra.run.smithi035.stderr: 48750K .......... .......... .......... .......... .......... 66% 278M 0s 2019-12-09T09:27:15.113 INFO:teuthology.orchestra.run.smithi035.stderr: 48800K .......... .......... .......... .......... .......... 66% 314M 0s 2019-12-09T09:27:15.113 INFO:teuthology.orchestra.run.smithi035.stderr: 48850K .......... .......... .......... .......... .......... 66% 311M 0s 2019-12-09T09:27:15.114 INFO:teuthology.orchestra.run.smithi035.stderr: 48900K .......... .......... .......... .......... .......... 66% 237M 0s 2019-12-09T09:27:15.114 INFO:teuthology.orchestra.run.smithi035.stderr: 48950K .......... .......... .......... .......... .......... 66% 247M 0s 2019-12-09T09:27:15.114 INFO:teuthology.orchestra.run.smithi035.stderr: 49000K .......... .......... .......... .......... .......... 66% 279M 0s 2019-12-09T09:27:15.114 INFO:teuthology.orchestra.run.smithi035.stderr: 49050K .......... .......... .......... .......... .......... 66% 372M 0s 2019-12-09T09:27:15.114 INFO:teuthology.orchestra.run.smithi035.stderr: 49100K .......... .......... .......... .......... .......... 66% 313M 0s 2019-12-09T09:27:15.114 INFO:teuthology.orchestra.run.smithi035.stderr: 49150K .......... .......... .......... .......... .......... 66% 214M 0s 2019-12-09T09:27:15.114 INFO:teuthology.orchestra.run.smithi035.stderr: 49200K .......... .......... .......... .......... .......... 66% 222M 0s 2019-12-09T09:27:15.115 INFO:teuthology.orchestra.run.smithi035.stderr: 49250K .......... .......... .......... .......... .......... 67% 277M 0s 2019-12-09T09:27:15.115 INFO:teuthology.orchestra.run.smithi035.stderr: 49300K .......... .......... .......... .......... .......... 67% 309M 0s 2019-12-09T09:27:15.115 INFO:teuthology.orchestra.run.smithi035.stderr: 49350K .......... .......... .......... .......... .......... 67% 285M 0s 2019-12-09T09:27:15.115 INFO:teuthology.orchestra.run.smithi035.stderr: 49400K .......... .......... .......... .......... .......... 67% 298M 0s 2019-12-09T09:27:15.115 INFO:teuthology.orchestra.run.smithi035.stderr: 49450K .......... .......... .......... .......... .......... 67% 294M 0s 2019-12-09T09:27:15.115 INFO:teuthology.orchestra.run.smithi035.stderr: 49500K .......... .......... .......... .......... .......... 67% 308M 0s 2019-12-09T09:27:15.116 INFO:teuthology.orchestra.run.smithi035.stderr: 49550K .......... .......... .......... .......... .......... 67% 275M 0s 2019-12-09T09:27:15.116 INFO:teuthology.orchestra.run.smithi035.stderr: 49600K .......... .......... .......... .......... .......... 67% 313M 0s 2019-12-09T09:27:15.116 INFO:teuthology.orchestra.run.smithi035.stderr: 49650K .......... .......... .......... .......... .......... 67% 266M 0s 2019-12-09T09:27:15.116 INFO:teuthology.orchestra.run.smithi035.stderr: 49700K .......... .......... .......... .......... .......... 67% 318M 0s 2019-12-09T09:27:15.116 INFO:teuthology.orchestra.run.smithi035.stderr: 49750K .......... .......... .......... .......... .......... 67% 210M 0s 2019-12-09T09:27:15.116 INFO:teuthology.orchestra.run.smithi035.stderr: 49800K .......... .......... .......... .......... .......... 67% 286M 0s 2019-12-09T09:27:15.116 INFO:teuthology.orchestra.run.smithi035.stderr: 49850K .......... .......... .......... .......... .......... 67% 288M 0s 2019-12-09T09:27:15.117 INFO:teuthology.orchestra.run.smithi035.stderr: 49900K .......... .......... .......... .......... .......... 67% 300M 0s 2019-12-09T09:27:15.117 INFO:teuthology.orchestra.run.smithi035.stderr: 49950K .......... .......... .......... .......... .......... 67% 253M 0s 2019-12-09T09:27:15.117 INFO:teuthology.orchestra.run.smithi035.stderr: 50000K .......... .......... .......... .......... .......... 68% 325M 0s 2019-12-09T09:27:15.117 INFO:teuthology.orchestra.run.smithi035.stderr: 50050K .......... .......... .......... .......... .......... 68% 275M 0s 2019-12-09T09:27:15.117 INFO:teuthology.orchestra.run.smithi035.stderr: 50100K .......... .......... .......... .......... .......... 68% 270M 0s 2019-12-09T09:27:15.117 INFO:teuthology.orchestra.run.smithi035.stderr: 50150K .......... .......... .......... .......... .......... 68% 253M 0s 2019-12-09T09:27:15.117 INFO:teuthology.orchestra.run.smithi035.stderr: 50200K .......... .......... .......... .......... .......... 68% 269M 0s 2019-12-09T09:27:15.118 INFO:teuthology.orchestra.run.smithi035.stderr: 50250K .......... .......... .......... .......... .......... 68% 321M 0s 2019-12-09T09:27:15.118 INFO:teuthology.orchestra.run.smithi035.stderr: 50300K .......... .......... .......... .......... .......... 68% 321M 0s 2019-12-09T09:27:15.118 INFO:teuthology.orchestra.run.smithi035.stderr: 50350K .......... .......... .......... .......... .......... 68% 283M 0s 2019-12-09T09:27:15.118 INFO:teuthology.orchestra.run.smithi035.stderr: 50400K .......... .......... .......... .......... .......... 68% 242M 0s 2019-12-09T09:27:15.118 INFO:teuthology.orchestra.run.smithi035.stderr: 50450K .......... .......... .......... .......... .......... 68% 358M 0s 2019-12-09T09:27:15.118 INFO:teuthology.orchestra.run.smithi035.stderr: 50500K .......... .......... .......... .......... .......... 68% 268M 0s 2019-12-09T09:27:15.118 INFO:teuthology.orchestra.run.smithi035.stderr: 50550K .......... .......... .......... .......... .......... 68% 214M 0s 2019-12-09T09:27:15.118 INFO:teuthology.orchestra.run.smithi035.stderr: 50600K .......... .......... .......... .......... .......... 68% 262M 0s 2019-12-09T09:27:15.119 INFO:teuthology.orchestra.run.smithi035.stderr: 50650K .......... .......... .......... .......... .......... 68% 257M 0s 2019-12-09T09:27:15.119 INFO:teuthology.orchestra.run.smithi035.stderr: 50700K .......... .......... .......... .......... .......... 68% 297M 0s 2019-12-09T09:27:15.119 INFO:teuthology.orchestra.run.smithi035.stderr: 50750K .......... .......... .......... .......... .......... 69% 276M 0s 2019-12-09T09:27:15.119 INFO:teuthology.orchestra.run.smithi035.stderr: 50800K .......... .......... .......... .......... .......... 69% 337M 0s 2019-12-09T09:27:15.119 INFO:teuthology.orchestra.run.smithi035.stderr: 50850K .......... .......... .......... .......... .......... 69% 304M 0s 2019-12-09T09:27:15.119 INFO:teuthology.orchestra.run.smithi035.stderr: 50900K .......... .......... .......... .......... .......... 69% 294M 0s 2019-12-09T09:27:15.119 INFO:teuthology.orchestra.run.smithi035.stderr: 50950K .......... .......... .......... .......... .......... 69% 194M 0s 2019-12-09T09:27:15.120 INFO:teuthology.orchestra.run.smithi035.stderr: 51000K .......... .......... .......... .......... .......... 69% 302M 0s 2019-12-09T09:27:15.120 INFO:teuthology.orchestra.run.smithi035.stderr: 51050K .......... .......... .......... .......... .......... 69% 329M 0s 2019-12-09T09:27:15.120 INFO:teuthology.orchestra.run.smithi035.stderr: 51100K .......... .......... .......... .......... .......... 69% 294M 0s 2019-12-09T09:27:15.120 INFO:teuthology.orchestra.run.smithi035.stderr: 51150K .......... .......... .......... .......... .......... 69% 270M 0s 2019-12-09T09:27:15.120 INFO:teuthology.orchestra.run.smithi035.stderr: 51200K .......... .......... .......... .......... .......... 69% 282M 0s 2019-12-09T09:27:15.120 INFO:teuthology.orchestra.run.smithi035.stderr: 51250K .......... .......... .......... .......... .......... 69% 281M 0s 2019-12-09T09:27:15.120 INFO:teuthology.orchestra.run.smithi035.stderr: 51300K .......... .......... .......... .......... .......... 69% 331M 0s 2019-12-09T09:27:15.121 INFO:teuthology.orchestra.run.smithi035.stderr: 51350K .......... .......... .......... .......... .......... 69% 268M 0s 2019-12-09T09:27:15.121 INFO:teuthology.orchestra.run.smithi035.stderr: 51400K .......... .......... .......... .......... .......... 69% 219M 0s 2019-12-09T09:27:15.121 INFO:teuthology.orchestra.run.smithi035.stderr: 51450K .......... .......... .......... .......... .......... 70% 294M 0s 2019-12-09T09:27:15.121 INFO:teuthology.orchestra.run.smithi035.stderr: 51500K .......... .......... .......... .......... .......... 70% 275M 0s 2019-12-09T09:27:15.121 INFO:teuthology.orchestra.run.smithi035.stderr: 51550K .......... .......... .......... .......... .......... 70% 286M 0s 2019-12-09T09:27:15.121 INFO:teuthology.orchestra.run.smithi035.stderr: 51600K .......... .......... .......... .......... .......... 70% 296M 0s 2019-12-09T09:27:15.121 INFO:teuthology.orchestra.run.smithi035.stderr: 51650K .......... .......... .......... .......... .......... 70% 288M 0s 2019-12-09T09:27:15.122 INFO:teuthology.orchestra.run.smithi035.stderr: 51700K .......... .......... .......... .......... .......... 70% 294M 0s 2019-12-09T09:27:15.122 INFO:teuthology.orchestra.run.smithi035.stderr: 51750K .......... .......... .......... .......... .......... 70% 304M 0s 2019-12-09T09:27:15.122 INFO:teuthology.orchestra.run.smithi035.stderr: 51800K .......... .......... .......... .......... .......... 70% 305M 0s 2019-12-09T09:27:15.122 INFO:teuthology.orchestra.run.smithi035.stderr: 51850K .......... .......... .......... .......... .......... 70% 252M 0s 2019-12-09T09:27:15.122 INFO:teuthology.orchestra.run.smithi035.stderr: 51900K .......... .......... .......... .......... .......... 70% 325M 0s 2019-12-09T09:27:15.122 INFO:teuthology.orchestra.run.smithi035.stderr: 51950K .......... .......... .......... .......... .......... 70% 263M 0s 2019-12-09T09:27:15.123 INFO:teuthology.orchestra.run.smithi035.stderr: 52000K .......... .......... .......... .......... .......... 70% 323M 0s 2019-12-09T09:27:15.123 INFO:teuthology.orchestra.run.smithi035.stderr: 52050K .......... .......... .......... .......... .......... 70% 297M 0s 2019-12-09T09:27:15.123 INFO:teuthology.orchestra.run.smithi035.stderr: 52100K .......... .......... .......... .......... .......... 70% 297M 0s 2019-12-09T09:27:15.123 INFO:teuthology.orchestra.run.smithi035.stderr: 52150K .......... .......... .......... .......... .......... 70% 202M 0s 2019-12-09T09:27:15.123 INFO:teuthology.orchestra.run.smithi035.stderr: 52200K .......... .......... .......... .......... .......... 71% 242M 0s 2019-12-09T09:27:15.123 INFO:teuthology.orchestra.run.smithi035.stderr: 52250K .......... .......... .......... .......... .......... 71% 341M 0s 2019-12-09T09:27:15.123 INFO:teuthology.orchestra.run.smithi035.stderr: 52300K .......... .......... .......... .......... .......... 71% 299M 0s 2019-12-09T09:27:15.124 INFO:teuthology.orchestra.run.smithi035.stderr: 52350K .......... .......... .......... .......... .......... 71% 274M 0s 2019-12-09T09:27:15.124 INFO:teuthology.orchestra.run.smithi035.stderr: 52400K .......... .......... .......... .......... .......... 71% 285M 0s 2019-12-09T09:27:15.124 INFO:teuthology.orchestra.run.smithi035.stderr: 52450K .......... .......... .......... .......... .......... 71% 308M 0s 2019-12-09T09:27:15.124 INFO:teuthology.orchestra.run.smithi035.stderr: 52500K .......... .......... .......... .......... .......... 71% 315M 0s 2019-12-09T09:27:15.124 INFO:teuthology.orchestra.run.smithi035.stderr: 52550K .......... .......... .......... .......... .......... 71% 261M 0s 2019-12-09T09:27:15.124 INFO:teuthology.orchestra.run.smithi035.stderr: 52600K .......... .......... .......... .......... .......... 71% 199M 0s 2019-12-09T09:27:15.124 INFO:teuthology.orchestra.run.smithi035.stderr: 52650K .......... .......... .......... .......... .......... 71% 270M 0s 2019-12-09T09:27:15.125 INFO:teuthology.orchestra.run.smithi035.stderr: 52700K .......... .......... .......... .......... .......... 71% 338M 0s 2019-12-09T09:27:15.125 INFO:teuthology.orchestra.run.smithi035.stderr: 52750K .......... .......... .......... .......... .......... 71% 308M 0s 2019-12-09T09:27:15.125 INFO:teuthology.orchestra.run.smithi035.stderr: 52800K .......... .......... .......... .......... .......... 71% 319M 0s 2019-12-09T09:27:15.125 INFO:teuthology.orchestra.run.smithi035.stderr: 52850K .......... .......... .......... .......... .......... 71% 242M 0s 2019-12-09T09:27:15.125 INFO:teuthology.orchestra.run.smithi035.stderr: 52900K .......... .......... .......... .......... .......... 71% 331M 0s 2019-12-09T09:27:15.125 INFO:teuthology.orchestra.run.smithi035.stderr: 52950K .......... .......... .......... .......... .......... 72% 220M 0s 2019-12-09T09:27:15.125 INFO:teuthology.orchestra.run.smithi035.stderr: 53000K .......... .......... .......... .......... .......... 72% 231M 0s 2019-12-09T09:27:15.125 INFO:teuthology.orchestra.run.smithi035.stderr: 53050K .......... .......... .......... .......... .......... 72% 325M 0s 2019-12-09T09:27:15.126 INFO:teuthology.orchestra.run.smithi035.stderr: 53100K .......... .......... .......... .......... .......... 72% 338M 0s 2019-12-09T09:27:15.126 INFO:teuthology.orchestra.run.smithi035.stderr: 53150K .......... .......... .......... .......... .......... 72% 296M 0s 2019-12-09T09:27:15.126 INFO:teuthology.orchestra.run.smithi035.stderr: 53200K .......... .......... .......... .......... .......... 72% 226M 0s 2019-12-09T09:27:15.126 INFO:teuthology.orchestra.run.smithi035.stderr: 53250K .......... .......... .......... .......... .......... 72% 244M 0s 2019-12-09T09:27:15.126 INFO:teuthology.orchestra.run.smithi035.stderr: 53300K .......... .......... .......... .......... .......... 72% 265M 0s 2019-12-09T09:27:15.126 INFO:teuthology.orchestra.run.smithi035.stderr: 53350K .......... .......... .......... .......... .......... 72% 228M 0s 2019-12-09T09:27:15.126 INFO:teuthology.orchestra.run.smithi035.stderr: 53400K .......... .......... .......... .......... .......... 72% 258M 0s 2019-12-09T09:27:15.127 INFO:teuthology.orchestra.run.smithi035.stderr: 53450K .......... .......... .......... .......... .......... 72% 260M 0s 2019-12-09T09:27:15.127 INFO:teuthology.orchestra.run.smithi035.stderr: 53500K .......... .......... .......... .......... .......... 72% 242M 0s 2019-12-09T09:27:15.127 INFO:teuthology.orchestra.run.smithi035.stderr: 53550K .......... .......... .......... .......... .......... 72% 218M 0s 2019-12-09T09:27:15.127 INFO:teuthology.orchestra.run.smithi035.stderr: 53600K .......... .......... .......... .......... .......... 72% 262M 0s 2019-12-09T09:27:15.127 INFO:teuthology.orchestra.run.smithi035.stderr: 53650K .......... .......... .......... .......... .......... 73% 245M 0s 2019-12-09T09:27:15.127 INFO:teuthology.orchestra.run.smithi035.stderr: 53700K .......... .......... .......... .......... .......... 73% 245M 0s 2019-12-09T09:27:15.127 INFO:teuthology.orchestra.run.smithi035.stderr: 53750K .......... .......... .......... .......... .......... 73% 220M 0s 2019-12-09T09:27:15.128 INFO:teuthology.orchestra.run.smithi035.stderr: 53800K .......... .......... .......... .......... .......... 73% 245M 0s 2019-12-09T09:27:15.128 INFO:teuthology.orchestra.run.smithi035.stderr: 53850K .......... .......... .......... .......... .......... 73% 268M 0s 2019-12-09T09:27:15.128 INFO:teuthology.orchestra.run.smithi035.stderr: 53900K .......... .......... .......... .......... .......... 73% 258M 0s 2019-12-09T09:27:15.128 INFO:teuthology.orchestra.run.smithi035.stderr: 53950K .......... .......... .......... .......... .......... 73% 212M 0s 2019-12-09T09:27:15.128 INFO:teuthology.orchestra.run.smithi035.stderr: 54000K .......... .......... .......... .......... .......... 73% 257M 0s 2019-12-09T09:27:15.128 INFO:teuthology.orchestra.run.smithi035.stderr: 54050K .......... .......... .......... .......... .......... 73% 274M 0s 2019-12-09T09:27:15.128 INFO:teuthology.orchestra.run.smithi035.stderr: 54100K .......... .......... .......... .......... .......... 73% 274M 0s 2019-12-09T09:27:15.129 INFO:teuthology.orchestra.run.smithi035.stderr: 54150K .......... .......... .......... .......... .......... 73% 234M 0s 2019-12-09T09:27:15.129 INFO:teuthology.orchestra.run.smithi035.stderr: 54200K .......... .......... .......... .......... .......... 73% 268M 0s 2019-12-09T09:27:15.129 INFO:teuthology.orchestra.run.smithi035.stderr: 54250K .......... .......... .......... .......... .......... 73% 255M 0s 2019-12-09T09:27:15.129 INFO:teuthology.orchestra.run.smithi035.stderr: 54300K .......... .......... .......... .......... .......... 73% 275M 0s 2019-12-09T09:27:15.129 INFO:teuthology.orchestra.run.smithi035.stderr: 54350K .......... .......... .......... .......... .......... 73% 280M 0s 2019-12-09T09:27:15.129 INFO:teuthology.orchestra.run.smithi035.stderr: 54400K .......... .......... .......... .......... .......... 74% 341M 0s 2019-12-09T09:27:15.129 INFO:teuthology.orchestra.run.smithi035.stderr: 54450K .......... .......... .......... .......... .......... 74% 351M 0s 2019-12-09T09:27:15.130 INFO:teuthology.orchestra.run.smithi035.stderr: 54500K .......... .......... .......... .......... .......... 74% 351M 0s 2019-12-09T09:27:15.130 INFO:teuthology.orchestra.run.smithi035.stderr: 54550K .......... .......... .......... .......... .......... 74% 291M 0s 2019-12-09T09:27:15.130 INFO:teuthology.orchestra.run.smithi035.stderr: 54600K .......... .......... .......... .......... .......... 74% 342M 0s 2019-12-09T09:27:15.130 INFO:teuthology.orchestra.run.smithi035.stderr: 54650K .......... .......... .......... .......... .......... 74% 372M 0s 2019-12-09T09:27:15.130 INFO:teuthology.orchestra.run.smithi035.stderr: 54700K .......... .......... .......... .......... .......... 74% 349M 0s 2019-12-09T09:27:15.130 INFO:teuthology.orchestra.run.smithi035.stderr: 54750K .......... .......... .......... .......... .......... 74% 282M 0s 2019-12-09T09:27:15.130 INFO:teuthology.orchestra.run.smithi035.stderr: 54800K .......... .......... .......... .......... .......... 74% 346M 0s 2019-12-09T09:27:15.130 INFO:teuthology.orchestra.run.smithi035.stderr: 54850K .......... .......... .......... .......... .......... 74% 353M 0s 2019-12-09T09:27:15.131 INFO:teuthology.orchestra.run.smithi035.stderr: 54900K .......... .......... .......... .......... .......... 74% 274M 0s 2019-12-09T09:27:15.131 INFO:teuthology.orchestra.run.smithi035.stderr: 54950K .......... .......... .......... .......... .......... 74% 206M 0s 2019-12-09T09:27:15.131 INFO:teuthology.orchestra.run.smithi035.stderr: 55000K .......... .......... .......... .......... .......... 74% 229M 0s 2019-12-09T09:27:15.131 INFO:teuthology.orchestra.run.smithi035.stderr: 55050K .......... .......... .......... .......... .......... 74% 235M 0s 2019-12-09T09:27:15.131 INFO:teuthology.orchestra.run.smithi035.stderr: 55100K .......... .......... .......... .......... .......... 74% 241M 0s 2019-12-09T09:27:15.131 INFO:teuthology.orchestra.run.smithi035.stderr: 55150K .......... .......... .......... .......... .......... 75% 201M 0s 2019-12-09T09:27:15.131 INFO:teuthology.orchestra.run.smithi035.stderr: 55200K .......... .......... .......... .......... .......... 75% 253M 0s 2019-12-09T09:27:15.132 INFO:teuthology.orchestra.run.smithi035.stderr: 55250K .......... .......... .......... .......... .......... 75% 255M 0s 2019-12-09T09:27:15.132 INFO:teuthology.orchestra.run.smithi035.stderr: 55300K .......... .......... .......... .......... .......... 75% 257M 0s 2019-12-09T09:27:15.132 INFO:teuthology.orchestra.run.smithi035.stderr: 55350K .......... .......... .......... .......... .......... 75% 221M 0s 2019-12-09T09:27:15.132 INFO:teuthology.orchestra.run.smithi035.stderr: 55400K .......... .......... .......... .......... .......... 75% 248M 0s 2019-12-09T09:27:15.132 INFO:teuthology.orchestra.run.smithi035.stderr: 55450K .......... .......... .......... .......... .......... 75% 260M 0s 2019-12-09T09:27:15.132 INFO:teuthology.orchestra.run.smithi035.stderr: 55500K .......... .......... .......... .......... .......... 75% 250M 0s 2019-12-09T09:27:15.132 INFO:teuthology.orchestra.run.smithi035.stderr: 55550K .......... .......... .......... .......... .......... 75% 189M 0s 2019-12-09T09:27:15.133 INFO:teuthology.orchestra.run.smithi035.stderr: 55600K .......... .......... .......... .......... .......... 75% 216M 0s 2019-12-09T09:27:15.133 INFO:teuthology.orchestra.run.smithi035.stderr: 55650K .......... .......... .......... .......... .......... 75% 215M 0s 2019-12-09T09:27:15.133 INFO:teuthology.orchestra.run.smithi035.stderr: 55700K .......... .......... .......... .......... .......... 75% 243M 0s 2019-12-09T09:27:15.133 INFO:teuthology.orchestra.run.smithi035.stderr: 55750K .......... .......... .......... .......... .......... 75% 206M 0s 2019-12-09T09:27:15.133 INFO:teuthology.orchestra.run.smithi035.stderr: 55800K .......... .......... .......... .......... .......... 75% 221M 0s 2019-12-09T09:27:15.133 INFO:teuthology.orchestra.run.smithi035.stderr: 55850K .......... .......... .......... .......... .......... 75% 215M 0s 2019-12-09T09:27:15.133 INFO:teuthology.orchestra.run.smithi035.stderr: 55900K .......... .......... .......... .......... .......... 76% 247M 0s 2019-12-09T09:27:15.134 INFO:teuthology.orchestra.run.smithi035.stderr: 55950K .......... .......... .......... .......... .......... 76% 175M 0s 2019-12-09T09:27:15.134 INFO:teuthology.orchestra.run.smithi035.stderr: 56000K .......... .......... .......... .......... .......... 76% 241M 0s 2019-12-09T09:27:15.134 INFO:teuthology.orchestra.run.smithi035.stderr: 56050K .......... .......... .......... .......... .......... 76% 259M 0s 2019-12-09T09:27:15.134 INFO:teuthology.orchestra.run.smithi035.stderr: 56100K .......... .......... .......... .......... .......... 76% 260M 0s 2019-12-09T09:27:15.134 INFO:teuthology.orchestra.run.smithi035.stderr: 56150K .......... .......... .......... .......... .......... 76% 224M 0s 2019-12-09T09:27:15.134 INFO:teuthology.orchestra.run.smithi035.stderr: 56200K .......... .......... .......... .......... .......... 76% 262M 0s 2019-12-09T09:27:15.134 INFO:teuthology.orchestra.run.smithi035.stderr: 56250K .......... .......... .......... .......... .......... 76% 251M 0s 2019-12-09T09:27:15.134 INFO:teuthology.orchestra.run.smithi035.stderr: 56300K .......... .......... .......... .......... .......... 76% 246M 0s 2019-12-09T09:27:15.135 INFO:teuthology.orchestra.run.smithi035.stderr: 56350K .......... .......... .......... .......... .......... 76% 218M 0s 2019-12-09T09:27:15.135 INFO:teuthology.orchestra.run.smithi035.stderr: 56400K .......... .......... .......... .......... .......... 76% 254M 0s 2019-12-09T09:27:15.135 INFO:teuthology.orchestra.run.smithi035.stderr: 56450K .......... .......... .......... .......... .......... 76% 257M 0s 2019-12-09T09:27:15.135 INFO:teuthology.orchestra.run.smithi035.stderr: 56500K .......... .......... .......... .......... .......... 76% 245M 0s 2019-12-09T09:27:15.135 INFO:teuthology.orchestra.run.smithi035.stderr: 56550K .......... .......... .......... .......... .......... 76% 225M 0s 2019-12-09T09:27:15.135 INFO:teuthology.orchestra.run.smithi035.stderr: 56600K .......... .......... .......... .......... .......... 77% 250M 0s 2019-12-09T09:27:15.135 INFO:teuthology.orchestra.run.smithi035.stderr: 56650K .......... .......... .......... .......... .......... 77% 245M 0s 2019-12-09T09:27:15.136 INFO:teuthology.orchestra.run.smithi035.stderr: 56700K .......... .......... .......... .......... .......... 77% 248M 0s 2019-12-09T09:27:15.136 INFO:teuthology.orchestra.run.smithi035.stderr: 56750K .......... .......... .......... .......... .......... 77% 203M 0s 2019-12-09T09:27:15.136 INFO:teuthology.orchestra.run.smithi035.stderr: 56800K .......... .......... .......... .......... .......... 77% 247M 0s 2019-12-09T09:27:15.136 INFO:teuthology.orchestra.run.smithi035.stderr: 56850K .......... .......... .......... .......... .......... 77% 281M 0s 2019-12-09T09:27:15.136 INFO:teuthology.orchestra.run.smithi035.stderr: 56900K .......... .......... .......... .......... .......... 77% 263M 0s 2019-12-09T09:27:15.136 INFO:teuthology.orchestra.run.smithi035.stderr: 56950K .......... .......... .......... .......... .......... 77% 220M 0s 2019-12-09T09:27:15.136 INFO:teuthology.orchestra.run.smithi035.stderr: 57000K .......... .......... .......... .......... .......... 77% 243M 0s 2019-12-09T09:27:15.137 INFO:teuthology.orchestra.run.smithi035.stderr: 57050K .......... .......... .......... .......... .......... 77% 260M 0s 2019-12-09T09:27:15.137 INFO:teuthology.orchestra.run.smithi035.stderr: 57100K .......... .......... .......... .......... .......... 77% 264M 0s 2019-12-09T09:27:15.137 INFO:teuthology.orchestra.run.smithi035.stderr: 57150K .......... .......... .......... .......... .......... 77% 231M 0s 2019-12-09T09:27:15.137 INFO:teuthology.orchestra.run.smithi035.stderr: 57200K .......... .......... .......... .......... .......... 77% 261M 0s 2019-12-09T09:27:15.137 INFO:teuthology.orchestra.run.smithi035.stderr: 57250K .......... .......... .......... .......... .......... 77% 262M 0s 2019-12-09T09:27:15.137 INFO:teuthology.orchestra.run.smithi035.stderr: 57300K .......... .......... .......... .......... .......... 77% 264M 0s 2019-12-09T09:27:15.137 INFO:teuthology.orchestra.run.smithi035.stderr: 57350K .......... .......... .......... .......... .......... 78% 215M 0s 2019-12-09T09:27:15.137 INFO:teuthology.orchestra.run.smithi035.stderr: 57400K .......... .......... .......... .......... .......... 78% 261M 0s 2019-12-09T09:27:15.138 INFO:teuthology.orchestra.run.smithi035.stderr: 57450K .......... .......... .......... .......... .......... 78% 269M 0s 2019-12-09T09:27:15.138 INFO:teuthology.orchestra.run.smithi035.stderr: 57500K .......... .......... .......... .......... .......... 78% 263M 0s 2019-12-09T09:27:15.138 INFO:teuthology.orchestra.run.smithi035.stderr: 57550K .......... .......... .......... .......... .......... 78% 214M 0s 2019-12-09T09:27:15.138 INFO:teuthology.orchestra.run.smithi035.stderr: 57600K .......... .......... .......... .......... .......... 78% 251M 0s 2019-12-09T09:27:15.138 INFO:teuthology.orchestra.run.smithi035.stderr: 57650K .......... .......... .......... .......... .......... 78% 261M 0s 2019-12-09T09:27:15.138 INFO:teuthology.orchestra.run.smithi035.stderr: 57700K .......... .......... .......... .......... .......... 78% 271M 0s 2019-12-09T09:27:15.138 INFO:teuthology.orchestra.run.smithi035.stderr: 57750K .......... .......... .......... .......... .......... 78% 199M 0s 2019-12-09T09:27:15.139 INFO:teuthology.orchestra.run.smithi035.stderr: 57800K .......... .......... .......... .......... .......... 78% 225M 0s 2019-12-09T09:27:15.139 INFO:teuthology.orchestra.run.smithi035.stderr: 57850K .......... .......... .......... .......... .......... 78% 212M 0s 2019-12-09T09:27:15.139 INFO:teuthology.orchestra.run.smithi035.stderr: 57900K .......... .......... .......... .......... .......... 78% 218M 0s 2019-12-09T09:27:15.139 INFO:teuthology.orchestra.run.smithi035.stderr: 57950K .......... .......... .......... .......... .......... 78% 212M 0s 2019-12-09T09:27:15.139 INFO:teuthology.orchestra.run.smithi035.stderr: 58000K .......... .......... .......... .......... .......... 78% 258M 0s 2019-12-09T09:27:15.139 INFO:teuthology.orchestra.run.smithi035.stderr: 58050K .......... .......... .......... .......... .......... 78% 245M 0s 2019-12-09T09:27:15.139 INFO:teuthology.orchestra.run.smithi035.stderr: 58100K .......... .......... .......... .......... .......... 79% 255M 0s 2019-12-09T09:27:15.139 INFO:teuthology.orchestra.run.smithi035.stderr: 58150K .......... .......... .......... .......... .......... 79% 214M 0s 2019-12-09T09:27:15.140 INFO:teuthology.orchestra.run.smithi035.stderr: 58200K .......... .......... .......... .......... .......... 79% 248M 0s 2019-12-09T09:27:15.140 INFO:teuthology.orchestra.run.smithi035.stderr: 58250K .......... .......... .......... .......... .......... 79% 248M 0s 2019-12-09T09:27:15.140 INFO:teuthology.orchestra.run.smithi035.stderr: 58300K .......... .......... .......... .......... .......... 79% 254M 0s 2019-12-09T09:27:15.140 INFO:teuthology.orchestra.run.smithi035.stderr: 58350K .......... .......... .......... .......... .......... 79% 211M 0s 2019-12-09T09:27:15.140 INFO:teuthology.orchestra.run.smithi035.stderr: 58400K .......... .......... .......... .......... .......... 79% 268M 0s 2019-12-09T09:27:15.140 INFO:teuthology.orchestra.run.smithi035.stderr: 58450K .......... .......... .......... .......... .......... 79% 253M 0s 2019-12-09T09:27:15.140 INFO:teuthology.orchestra.run.smithi035.stderr: 58500K .......... .......... .......... .......... .......... 79% 249M 0s 2019-12-09T09:27:15.141 INFO:teuthology.orchestra.run.smithi035.stderr: 58550K .......... .......... .......... .......... .......... 79% 207M 0s 2019-12-09T09:27:15.141 INFO:teuthology.orchestra.run.smithi035.stderr: 58600K .......... .......... .......... .......... .......... 79% 255M 0s 2019-12-09T09:27:15.141 INFO:teuthology.orchestra.run.smithi035.stderr: 58650K .......... .......... .......... .......... .......... 79% 259M 0s 2019-12-09T09:27:15.141 INFO:teuthology.orchestra.run.smithi035.stderr: 58700K .......... .......... .......... .......... .......... 79% 254M 0s 2019-12-09T09:27:15.141 INFO:teuthology.orchestra.run.smithi035.stderr: 58750K .......... .......... .......... .......... .......... 79% 242M 0s 2019-12-09T09:27:15.141 INFO:teuthology.orchestra.run.smithi035.stderr: 58800K .......... .......... .......... .......... .......... 80% 227M 0s 2019-12-09T09:27:15.141 INFO:teuthology.orchestra.run.smithi035.stderr: 58850K .......... .......... .......... .......... .......... 80% 254M 0s 2019-12-09T09:27:15.142 INFO:teuthology.orchestra.run.smithi035.stderr: 58900K .......... .......... .......... .......... .......... 80% 271M 0s 2019-12-09T09:27:15.142 INFO:teuthology.orchestra.run.smithi035.stderr: 58950K .......... .......... .......... .......... .......... 80% 204M 0s 2019-12-09T09:27:15.142 INFO:teuthology.orchestra.run.smithi035.stderr: 59000K .......... .......... .......... .......... .......... 80% 232M 0s 2019-12-09T09:27:15.142 INFO:teuthology.orchestra.run.smithi035.stderr: 59050K .......... .......... .......... .......... .......... 80% 224M 0s 2019-12-09T09:27:15.142 INFO:teuthology.orchestra.run.smithi035.stderr: 59100K .......... .......... .......... .......... .......... 80% 249M 0s 2019-12-09T09:27:15.142 INFO:teuthology.orchestra.run.smithi035.stderr: 59150K .......... .......... .......... .......... .......... 80% 217M 0s 2019-12-09T09:27:15.142 INFO:teuthology.orchestra.run.smithi035.stderr: 59200K .......... .......... .......... .......... .......... 80% 281M 0s 2019-12-09T09:27:15.142 INFO:teuthology.orchestra.run.smithi035.stderr: 59250K .......... .......... .......... .......... .......... 80% 232M 0s 2019-12-09T09:27:15.143 INFO:teuthology.orchestra.run.smithi035.stderr: 59300K .......... .......... .......... .......... .......... 80% 240M 0s 2019-12-09T09:27:15.143 INFO:teuthology.orchestra.run.smithi035.stderr: 59350K .......... .......... .......... .......... .......... 80% 214M 0s 2019-12-09T09:27:15.143 INFO:teuthology.orchestra.run.smithi035.stderr: 59400K .......... .......... .......... .......... .......... 80% 235M 0s 2019-12-09T09:27:15.143 INFO:teuthology.orchestra.run.smithi035.stderr: 59450K .......... .......... .......... .......... .......... 80% 250M 0s 2019-12-09T09:27:15.143 INFO:teuthology.orchestra.run.smithi035.stderr: 59500K .......... .......... .......... .......... .......... 80% 247M 0s 2019-12-09T09:27:15.143 INFO:teuthology.orchestra.run.smithi035.stderr: 59550K .......... .......... .......... .......... .......... 81% 229M 0s 2019-12-09T09:27:15.143 INFO:teuthology.orchestra.run.smithi035.stderr: 59600K .......... .......... .......... .......... .......... 81% 175M 0s 2019-12-09T09:27:15.144 INFO:teuthology.orchestra.run.smithi035.stderr: 59650K .......... .......... .......... .......... .......... 81% 206M 0s 2019-12-09T09:27:15.144 INFO:teuthology.orchestra.run.smithi035.stderr: 59700K .......... .......... .......... .......... .......... 81% 270M 0s 2019-12-09T09:27:15.144 INFO:teuthology.orchestra.run.smithi035.stderr: 59750K .......... .......... .......... .......... .......... 81% 190M 0s 2019-12-09T09:27:15.144 INFO:teuthology.orchestra.run.smithi035.stderr: 59800K .......... .......... .......... .......... .......... 81% 254M 0s 2019-12-09T09:27:15.144 INFO:teuthology.orchestra.run.smithi035.stderr: 59850K .......... .......... .......... .......... .......... 81% 238M 0s 2019-12-09T09:27:15.144 INFO:teuthology.orchestra.run.smithi035.stderr: 59900K .......... .......... .......... .......... .......... 81% 241M 0s 2019-12-09T09:27:15.144 INFO:teuthology.orchestra.run.smithi035.stderr: 59950K .......... .......... .......... .......... .......... 81% 228M 0s 2019-12-09T09:27:15.145 INFO:teuthology.orchestra.run.smithi035.stderr: 60000K .......... .......... .......... .......... .......... 81% 229M 0s 2019-12-09T09:27:15.145 INFO:teuthology.orchestra.run.smithi035.stderr: 60050K .......... .......... .......... .......... .......... 81% 219M 0s 2019-12-09T09:27:15.145 INFO:teuthology.orchestra.run.smithi035.stderr: 60100K .......... .......... .......... .......... .......... 81% 241M 0s 2019-12-09T09:27:15.145 INFO:teuthology.orchestra.run.smithi035.stderr: 60150K .......... .......... .......... .......... .......... 81% 227M 0s 2019-12-09T09:27:15.145 INFO:teuthology.orchestra.run.smithi035.stderr: 60200K .......... .......... .......... .......... .......... 81% 221M 0s 2019-12-09T09:27:15.145 INFO:teuthology.orchestra.run.smithi035.stderr: 60250K .......... .......... .......... .......... .......... 81% 228M 0s 2019-12-09T09:27:15.145 INFO:teuthology.orchestra.run.smithi035.stderr: 60300K .......... .......... .......... .......... .......... 82% 349M 0s 2019-12-09T09:27:15.145 INFO:teuthology.orchestra.run.smithi035.stderr: 60350K .......... .......... .......... .......... .......... 82% 262M 0s 2019-12-09T09:27:15.146 INFO:teuthology.orchestra.run.smithi035.stderr: 60400K .......... .......... .......... .......... .......... 82% 282M 0s 2019-12-09T09:27:15.146 INFO:teuthology.orchestra.run.smithi035.stderr: 60450K .......... .......... .......... .......... .......... 82% 333M 0s 2019-12-09T09:27:15.146 INFO:teuthology.orchestra.run.smithi035.stderr: 60500K .......... .......... .......... .......... .......... 82% 303M 0s 2019-12-09T09:27:15.146 INFO:teuthology.orchestra.run.smithi035.stderr: 60550K .......... .......... .......... .......... .......... 82% 272M 0s 2019-12-09T09:27:15.146 INFO:teuthology.orchestra.run.smithi035.stderr: 60600K .......... .......... .......... .......... .......... 82% 382M 0s 2019-12-09T09:27:15.146 INFO:teuthology.orchestra.run.smithi035.stderr: 60650K .......... .......... .......... .......... .......... 82% 363M 0s 2019-12-09T09:27:15.146 INFO:teuthology.orchestra.run.smithi035.stderr: 60700K .......... .......... .......... .......... .......... 82% 312M 0s 2019-12-09T09:27:15.147 INFO:teuthology.orchestra.run.smithi035.stderr: 60750K .......... .......... .......... .......... .......... 82% 281M 0s 2019-12-09T09:27:15.147 INFO:teuthology.orchestra.run.smithi035.stderr: 60800K .......... .......... .......... .......... .......... 82% 383M 0s 2019-12-09T09:27:15.147 INFO:teuthology.orchestra.run.smithi035.stderr: 60850K .......... .......... .......... .......... .......... 82% 321M 0s 2019-12-09T09:27:15.147 INFO:teuthology.orchestra.run.smithi035.stderr: 60900K .......... .......... .......... .......... .......... 82% 312M 0s 2019-12-09T09:27:15.147 INFO:teuthology.orchestra.run.smithi035.stderr: 60950K .......... .......... .......... .......... .......... 82% 287M 0s 2019-12-09T09:27:15.147 INFO:teuthology.orchestra.run.smithi035.stderr: 61000K .......... .......... .......... .......... .......... 82% 324M 0s 2019-12-09T09:27:15.147 INFO:teuthology.orchestra.run.smithi035.stderr: 61050K .......... .......... .......... .......... .......... 83% 327M 0s 2019-12-09T09:27:15.148 INFO:teuthology.orchestra.run.smithi035.stderr: 61100K .......... .......... .......... .......... .......... 83% 369M 0s 2019-12-09T09:27:15.148 INFO:teuthology.orchestra.run.smithi035.stderr: 61150K .......... .......... .......... .......... .......... 83% 269M 0s 2019-12-09T09:27:15.148 INFO:teuthology.orchestra.run.smithi035.stderr: 61200K .......... .......... .......... .......... .......... 83% 358M 0s 2019-12-09T09:27:15.148 INFO:teuthology.orchestra.run.smithi035.stderr: 61250K .......... .......... .......... .......... .......... 83% 243M 0s 2019-12-09T09:27:15.148 INFO:teuthology.orchestra.run.smithi035.stderr: 61300K .......... .......... .......... .......... .......... 83% 254M 0s 2019-12-09T09:27:15.148 INFO:teuthology.orchestra.run.smithi035.stderr: 61350K .......... .......... .......... .......... .......... 83% 249M 0s 2019-12-09T09:27:15.148 INFO:teuthology.orchestra.run.smithi035.stderr: 61400K .......... .......... .......... .......... .......... 83% 313M 0s 2019-12-09T09:27:15.148 INFO:teuthology.orchestra.run.smithi035.stderr: 61450K .......... .......... .......... .......... .......... 83% 274M 0s 2019-12-09T09:27:15.149 INFO:teuthology.orchestra.run.smithi035.stderr: 61500K .......... .......... .......... .......... .......... 83% 299M 0s 2019-12-09T09:27:15.149 INFO:teuthology.orchestra.run.smithi035.stderr: 61550K .......... .......... .......... .......... .......... 83% 244M 0s 2019-12-09T09:27:15.149 INFO:teuthology.orchestra.run.smithi035.stderr: 61600K .......... .......... .......... .......... .......... 83% 330M 0s 2019-12-09T09:27:15.149 INFO:teuthology.orchestra.run.smithi035.stderr: 61650K .......... .......... .......... .......... .......... 83% 295M 0s 2019-12-09T09:27:15.149 INFO:teuthology.orchestra.run.smithi035.stderr: 61700K .......... .......... .......... .......... .......... 83% 267M 0s 2019-12-09T09:27:15.149 INFO:teuthology.orchestra.run.smithi035.stderr: 61750K .......... .......... .......... .......... .......... 84% 290M 0s 2019-12-09T09:27:15.149 INFO:teuthology.orchestra.run.smithi035.stderr: 61800K .......... .......... .......... .......... .......... 84% 249M 0s 2019-12-09T09:27:15.150 INFO:teuthology.orchestra.run.smithi035.stderr: 61850K .......... .......... .......... .......... .......... 84% 309M 0s 2019-12-09T09:27:15.150 INFO:teuthology.orchestra.run.smithi035.stderr: 61900K .......... .......... .......... .......... .......... 84% 277M 0s 2019-12-09T09:27:15.150 INFO:teuthology.orchestra.run.smithi035.stderr: 61950K .......... .......... .......... .......... .......... 84% 227M 0s 2019-12-09T09:27:15.150 INFO:teuthology.orchestra.run.smithi035.stderr: 62000K .......... .......... .......... .......... .......... 84% 320M 0s 2019-12-09T09:27:15.150 INFO:teuthology.orchestra.run.smithi035.stderr: 62050K .......... .......... .......... .......... .......... 84% 350M 0s 2019-12-09T09:27:15.150 INFO:teuthology.orchestra.run.smithi035.stderr: 62100K .......... .......... .......... .......... .......... 84% 280M 0s 2019-12-09T09:27:15.150 INFO:teuthology.orchestra.run.smithi035.stderr: 62150K .......... .......... .......... .......... .......... 84% 288M 0s 2019-12-09T09:27:15.151 INFO:teuthology.orchestra.run.smithi035.stderr: 62200K .......... .......... .......... .......... .......... 84% 393M 0s 2019-12-09T09:27:15.151 INFO:teuthology.orchestra.run.smithi035.stderr: 62250K .......... .......... .......... .......... .......... 84% 380M 0s 2019-12-09T09:27:15.151 INFO:teuthology.orchestra.run.smithi035.stderr: 62300K .......... .......... .......... .......... .......... 84% 331M 0s 2019-12-09T09:27:15.151 INFO:teuthology.orchestra.run.smithi035.stderr: 62350K .......... .......... .......... .......... .......... 84% 255M 0s 2019-12-09T09:27:15.151 INFO:teuthology.orchestra.run.smithi035.stderr: 62400K .......... .......... .......... .......... .......... 84% 326M 0s 2019-12-09T09:27:15.151 INFO:teuthology.orchestra.run.smithi035.stderr: 62450K .......... .......... .......... .......... .......... 84% 309M 0s 2019-12-09T09:27:15.151 INFO:teuthology.orchestra.run.smithi035.stderr: 62500K .......... .......... .......... .......... .......... 85% 319M 0s 2019-12-09T09:27:15.151 INFO:teuthology.orchestra.run.smithi035.stderr: 62550K .......... .......... .......... .......... .......... 85% 287M 0s 2019-12-09T09:27:15.152 INFO:teuthology.orchestra.run.smithi035.stderr: 62600K .......... .......... .......... .......... .......... 85% 358M 0s 2019-12-09T09:27:15.152 INFO:teuthology.orchestra.run.smithi035.stderr: 62650K .......... .......... .......... .......... .......... 85% 296M 0s 2019-12-09T09:27:15.152 INFO:teuthology.orchestra.run.smithi035.stderr: 62700K .......... .......... .......... .......... .......... 85% 295M 0s 2019-12-09T09:27:15.152 INFO:teuthology.orchestra.run.smithi035.stderr: 62750K .......... .......... .......... .......... .......... 85% 258M 0s 2019-12-09T09:27:15.152 INFO:teuthology.orchestra.run.smithi035.stderr: 62800K .......... .......... .......... .......... .......... 85% 252M 0s 2019-12-09T09:27:15.152 INFO:teuthology.orchestra.run.smithi035.stderr: 62850K .......... .......... .......... .......... .......... 85% 344M 0s 2019-12-09T09:27:15.152 INFO:teuthology.orchestra.run.smithi035.stderr: 62900K .......... .......... .......... .......... .......... 85% 374M 0s 2019-12-09T09:27:15.153 INFO:teuthology.orchestra.run.smithi035.stderr: 62950K .......... .......... .......... .......... .......... 85% 318M 0s 2019-12-09T09:27:15.153 INFO:teuthology.orchestra.run.smithi035.stderr: 63000K .......... .......... .......... .......... .......... 85% 259M 0s 2019-12-09T09:27:15.153 INFO:teuthology.orchestra.run.smithi035.stderr: 63050K .......... .......... .......... .......... .......... 85% 307M 0s 2019-12-09T09:27:15.153 INFO:teuthology.orchestra.run.smithi035.stderr: 63100K .......... .......... .......... .......... .......... 85% 303M 0s 2019-12-09T09:27:15.153 INFO:teuthology.orchestra.run.smithi035.stderr: 63150K .......... .......... .......... .......... .......... 85% 256M 0s 2019-12-09T09:27:15.153 INFO:teuthology.orchestra.run.smithi035.stderr: 63200K .......... .......... .......... .......... .......... 85% 350M 0s 2019-12-09T09:27:15.153 INFO:teuthology.orchestra.run.smithi035.stderr: 63250K .......... .......... .......... .......... .......... 86% 309M 0s 2019-12-09T09:27:15.154 INFO:teuthology.orchestra.run.smithi035.stderr: 63300K .......... .......... .......... .......... .......... 86% 297M 0s 2019-12-09T09:27:15.154 INFO:teuthology.orchestra.run.smithi035.stderr: 63350K .......... .......... .......... .......... .......... 86% 240M 0s 2019-12-09T09:27:15.154 INFO:teuthology.orchestra.run.smithi035.stderr: 63400K .......... .......... .......... .......... .......... 86% 320M 0s 2019-12-09T09:27:15.154 INFO:teuthology.orchestra.run.smithi035.stderr: 63450K .......... .......... .......... .......... .......... 86% 316M 0s 2019-12-09T09:27:15.154 INFO:teuthology.orchestra.run.smithi035.stderr: 63500K .......... .......... .......... .......... .......... 86% 223M 0s 2019-12-09T09:27:15.154 INFO:teuthology.orchestra.run.smithi035.stderr: 63550K .......... .......... .......... .......... .......... 86% 290M 0s 2019-12-09T09:27:15.155 INFO:teuthology.orchestra.run.smithi035.stderr: 63600K .......... .......... .......... .......... .......... 86% 306M 0s 2019-12-09T09:27:15.155 INFO:teuthology.orchestra.run.smithi035.stderr: 63650K .......... .......... .......... .......... .......... 86% 335M 0s 2019-12-09T09:27:15.155 INFO:teuthology.orchestra.run.smithi035.stderr: 63700K .......... .......... .......... .......... .......... 86% 323M 0s 2019-12-09T09:27:15.155 INFO:teuthology.orchestra.run.smithi035.stderr: 63750K .......... .......... .......... .......... .......... 86% 276M 0s 2019-12-09T09:27:15.155 INFO:teuthology.orchestra.run.smithi035.stderr: 63800K .......... .......... .......... .......... .......... 86% 367M 0s 2019-12-09T09:27:15.155 INFO:teuthology.orchestra.run.smithi035.stderr: 63850K .......... .......... .......... .......... .......... 86% 342M 0s 2019-12-09T09:27:15.155 INFO:teuthology.orchestra.run.smithi035.stderr: 63900K .......... .......... .......... .......... .......... 86% 294M 0s 2019-12-09T09:27:15.156 INFO:teuthology.orchestra.run.smithi035.stderr: 63950K .......... .......... .......... .......... .......... 87% 331M 0s 2019-12-09T09:27:15.156 INFO:teuthology.orchestra.run.smithi035.stderr: 64000K .......... .......... .......... .......... .......... 87% 270M 0s 2019-12-09T09:27:15.156 INFO:teuthology.orchestra.run.smithi035.stderr: 64050K .......... .......... .......... .......... .......... 87% 233M 0s 2019-12-09T09:27:15.156 INFO:teuthology.orchestra.run.smithi035.stderr: 64100K .......... .......... .......... .......... .......... 87% 227M 0s 2019-12-09T09:27:15.156 INFO:teuthology.orchestra.run.smithi035.stderr: 64150K .......... .......... .......... .......... .......... 87% 177M 0s 2019-12-09T09:27:15.156 INFO:teuthology.orchestra.run.smithi035.stderr: 64200K .......... .......... .......... .......... .......... 87% 226M 0s 2019-12-09T09:27:15.156 INFO:teuthology.orchestra.run.smithi035.stderr: 64250K .......... .......... .......... .......... .......... 87% 204M 0s 2019-12-09T09:27:15.156 INFO:teuthology.orchestra.run.smithi035.stderr: 64300K .......... .......... .......... .......... .......... 87% 299M 0s 2019-12-09T09:27:15.157 INFO:teuthology.orchestra.run.smithi035.stderr: 64350K .......... .......... .......... .......... .......... 87% 202M 0s 2019-12-09T09:27:15.157 INFO:teuthology.orchestra.run.smithi035.stderr: 64400K .......... .......... .......... .......... .......... 87% 221M 0s 2019-12-09T09:27:15.157 INFO:teuthology.orchestra.run.smithi035.stderr: 64450K .......... .......... .......... .......... .......... 87% 245M 0s 2019-12-09T09:27:15.157 INFO:teuthology.orchestra.run.smithi035.stderr: 64500K .......... .......... .......... .......... .......... 87% 315M 0s 2019-12-09T09:27:15.157 INFO:teuthology.orchestra.run.smithi035.stderr: 64550K .......... .......... .......... .......... .......... 87% 264M 0s 2019-12-09T09:27:15.157 INFO:teuthology.orchestra.run.smithi035.stderr: 64600K .......... .......... .......... .......... .......... 87% 264M 0s 2019-12-09T09:27:15.157 INFO:teuthology.orchestra.run.smithi035.stderr: 64650K .......... .......... .......... .......... .......... 87% 297M 0s 2019-12-09T09:27:15.158 INFO:teuthology.orchestra.run.smithi035.stderr: 64700K .......... .......... .......... .......... .......... 88% 278M 0s 2019-12-09T09:27:15.158 INFO:teuthology.orchestra.run.smithi035.stderr: 64750K .......... .......... .......... .......... .......... 88% 217M 0s 2019-12-09T09:27:15.158 INFO:teuthology.orchestra.run.smithi035.stderr: 64800K .......... .......... .......... .......... .......... 88% 286M 0s 2019-12-09T09:27:15.158 INFO:teuthology.orchestra.run.smithi035.stderr: 64850K .......... .......... .......... .......... .......... 88% 303M 0s 2019-12-09T09:27:15.158 INFO:teuthology.orchestra.run.smithi035.stderr: 64900K .......... .......... .......... .......... .......... 88% 279M 0s 2019-12-09T09:27:15.158 INFO:teuthology.orchestra.run.smithi035.stderr: 64950K .......... .......... .......... .......... .......... 88% 246M 0s 2019-12-09T09:27:15.158 INFO:teuthology.orchestra.run.smithi035.stderr: 65000K .......... .......... .......... .......... .......... 88% 301M 0s 2019-12-09T09:27:15.159 INFO:teuthology.orchestra.run.smithi035.stderr: 65050K .......... .......... .......... .......... .......... 88% 358M 0s 2019-12-09T09:27:15.159 INFO:teuthology.orchestra.run.smithi035.stderr: 65100K .......... .......... .......... .......... .......... 88% 284M 0s 2019-12-09T09:27:15.159 INFO:teuthology.orchestra.run.smithi035.stderr: 65150K .......... .......... .......... .......... .......... 88% 259M 0s 2019-12-09T09:27:15.159 INFO:teuthology.orchestra.run.smithi035.stderr: 65200K .......... .......... .......... .......... .......... 88% 296M 0s 2019-12-09T09:27:15.159 INFO:teuthology.orchestra.run.smithi035.stderr: 65250K .......... .......... .......... .......... .......... 88% 348M 0s 2019-12-09T09:27:15.159 INFO:teuthology.orchestra.run.smithi035.stderr: 65300K .......... .......... .......... .......... .......... 88% 362M 0s 2019-12-09T09:27:15.159 INFO:teuthology.orchestra.run.smithi035.stderr: 65350K .......... .......... .......... .......... .......... 88% 291M 0s 2019-12-09T09:27:15.160 INFO:teuthology.orchestra.run.smithi035.stderr: 65400K .......... .......... .......... .......... .......... 88% 324M 0s 2019-12-09T09:27:15.160 INFO:teuthology.orchestra.run.smithi035.stderr: 65450K .......... .......... .......... .......... .......... 89% 313M 0s 2019-12-09T09:27:15.160 INFO:teuthology.orchestra.run.smithi035.stderr: 65500K .......... .......... .......... .......... .......... 89% 314M 0s 2019-12-09T09:27:15.160 INFO:teuthology.orchestra.run.smithi035.stderr: 65550K .......... .......... .......... .......... .......... 89% 292M 0s 2019-12-09T09:27:15.160 INFO:teuthology.orchestra.run.smithi035.stderr: 65600K .......... .......... .......... .......... .......... 89% 381M 0s 2019-12-09T09:27:15.161 INFO:teuthology.orchestra.run.smithi035.stderr: 65650K .......... .......... .......... .......... .......... 89% 345M 0s 2019-12-09T09:27:15.161 INFO:teuthology.orchestra.run.smithi035.stderr: 65700K .......... .......... .......... .......... .......... 89% 313M 0s 2019-12-09T09:27:15.161 INFO:teuthology.orchestra.run.smithi035.stderr: 65750K .......... .......... .......... .......... .......... 89% 271M 0s 2019-12-09T09:27:15.161 INFO:teuthology.orchestra.run.smithi035.stderr: 65800K .......... .......... .......... .......... .......... 89% 335M 0s 2019-12-09T09:27:15.161 INFO:teuthology.orchestra.run.smithi035.stderr: 65850K .......... .......... .......... .......... .......... 89% 285M 0s 2019-12-09T09:27:15.161 INFO:teuthology.orchestra.run.smithi035.stderr: 65900K .......... .......... .......... .......... .......... 89% 287M 0s 2019-12-09T09:27:15.161 INFO:teuthology.orchestra.run.smithi035.stderr: 65950K .......... .......... .......... .......... .......... 89% 293M 0s 2019-12-09T09:27:15.162 INFO:teuthology.orchestra.run.smithi035.stderr: 66000K .......... .......... .......... .......... .......... 89% 280M 0s 2019-12-09T09:27:15.162 INFO:teuthology.orchestra.run.smithi035.stderr: 66050K .......... .......... .......... .......... .......... 89% 299M 0s 2019-12-09T09:27:15.162 INFO:teuthology.orchestra.run.smithi035.stderr: 66100K .......... .......... .......... .......... .......... 89% 345M 0s 2019-12-09T09:27:15.162 INFO:teuthology.orchestra.run.smithi035.stderr: 66150K .......... .......... .......... .......... .......... 89% 230M 0s 2019-12-09T09:27:15.162 INFO:teuthology.orchestra.run.smithi035.stderr: 66200K .......... .......... .......... .......... .......... 90% 335M 0s 2019-12-09T09:27:15.162 INFO:teuthology.orchestra.run.smithi035.stderr: 66250K .......... .......... .......... .......... .......... 90% 275M 0s 2019-12-09T09:27:15.162 INFO:teuthology.orchestra.run.smithi035.stderr: 66300K .......... .......... .......... .......... .......... 90% 351M 0s 2019-12-09T09:27:15.163 INFO:teuthology.orchestra.run.smithi035.stderr: 66350K .......... .......... .......... .......... .......... 90% 253M 0s 2019-12-09T09:27:15.163 INFO:teuthology.orchestra.run.smithi035.stderr: 66400K .......... .......... .......... .......... .......... 90% 322M 0s 2019-12-09T09:27:15.163 INFO:teuthology.orchestra.run.smithi035.stderr: 66450K .......... .......... .......... .......... .......... 90% 298M 0s 2019-12-09T09:27:15.163 INFO:teuthology.orchestra.run.smithi035.stderr: 66500K .......... .......... .......... .......... .......... 90% 381M 0s 2019-12-09T09:27:15.163 INFO:teuthology.orchestra.run.smithi035.stderr: 66550K .......... .......... .......... .......... .......... 90% 283M 0s 2019-12-09T09:27:15.163 INFO:teuthology.orchestra.run.smithi035.stderr: 66600K .......... .......... .......... .......... .......... 90% 286M 0s 2019-12-09T09:27:15.163 INFO:teuthology.orchestra.run.smithi035.stderr: 66650K .......... .......... .......... .......... .......... 90% 288M 0s 2019-12-09T09:27:15.163 INFO:teuthology.orchestra.run.smithi035.stderr: 66700K .......... .......... .......... .......... .......... 90% 329M 0s 2019-12-09T09:27:15.164 INFO:teuthology.orchestra.run.smithi035.stderr: 66750K .......... .......... .......... .......... .......... 90% 248M 0s 2019-12-09T09:27:15.164 INFO:teuthology.orchestra.run.smithi035.stderr: 66800K .......... .......... .......... .......... .......... 90% 324M 0s 2019-12-09T09:27:15.164 INFO:teuthology.orchestra.run.smithi035.stderr: 66850K .......... .......... .......... .......... .......... 90% 301M 0s 2019-12-09T09:27:15.164 INFO:teuthology.orchestra.run.smithi035.stderr: 66900K .......... .......... .......... .......... .......... 91% 310M 0s 2019-12-09T09:27:15.164 INFO:teuthology.orchestra.run.smithi035.stderr: 66950K .......... .......... .......... .......... .......... 91% 264M 0s 2019-12-09T09:27:15.164 INFO:teuthology.orchestra.run.smithi035.stderr: 67000K .......... .......... .......... .......... .......... 91% 376M 0s 2019-12-09T09:27:15.164 INFO:teuthology.orchestra.run.smithi035.stderr: 67050K .......... .......... .......... .......... .......... 91% 327M 0s 2019-12-09T09:27:15.165 INFO:teuthology.orchestra.run.smithi035.stderr: 67100K .......... .......... .......... .......... .......... 91% 298M 0s 2019-12-09T09:27:15.165 INFO:teuthology.orchestra.run.smithi035.stderr: 67150K .......... .......... .......... .......... .......... 91% 308M 0s 2019-12-09T09:27:15.165 INFO:teuthology.orchestra.run.smithi035.stderr: 67200K .......... .......... .......... .......... .......... 91% 287M 0s 2019-12-09T09:27:15.165 INFO:teuthology.orchestra.run.smithi035.stderr: 67250K .......... .......... .......... .......... .......... 91% 314M 0s 2019-12-09T09:27:15.165 INFO:teuthology.orchestra.run.smithi035.stderr: 67300K .......... .......... .......... .......... .......... 91% 293M 0s 2019-12-09T09:27:15.165 INFO:teuthology.orchestra.run.smithi035.stderr: 67350K .......... .......... .......... .......... .......... 91% 315M 0s 2019-12-09T09:27:15.165 INFO:teuthology.orchestra.run.smithi035.stderr: 67400K .......... .......... .......... .......... .......... 91% 346M 0s 2019-12-09T09:27:15.166 INFO:teuthology.orchestra.run.smithi035.stderr: 67450K .......... .......... .......... .......... .......... 91% 274M 0s 2019-12-09T09:27:15.166 INFO:teuthology.orchestra.run.smithi035.stderr: 67500K .......... .......... .......... .......... .......... 91% 284M 0s 2019-12-09T09:27:15.166 INFO:teuthology.orchestra.run.smithi035.stderr: 67550K .......... .......... .......... .......... .......... 91% 303M 0s 2019-12-09T09:27:15.166 INFO:teuthology.orchestra.run.smithi035.stderr: 67600K .......... .......... .......... .......... .......... 91% 271M 0s 2019-12-09T09:27:15.166 INFO:teuthology.orchestra.run.smithi035.stderr: 67650K .......... .......... .......... .......... .......... 92% 251M 0s 2019-12-09T09:27:15.166 INFO:teuthology.orchestra.run.smithi035.stderr: 67700K .......... .......... .......... .......... .......... 92% 267M 0s 2019-12-09T09:27:15.167 INFO:teuthology.orchestra.run.smithi035.stderr: 67750K .......... .......... .......... .......... .......... 92% 263M 0s 2019-12-09T09:27:15.167 INFO:teuthology.orchestra.run.smithi035.stderr: 67800K .......... .......... .......... .......... .......... 92% 325M 0s 2019-12-09T09:27:15.167 INFO:teuthology.orchestra.run.smithi035.stderr: 67850K .......... .......... .......... .......... .......... 92% 321M 0s 2019-12-09T09:27:15.167 INFO:teuthology.orchestra.run.smithi035.stderr: 67900K .......... .......... .......... .......... .......... 92% 336M 0s 2019-12-09T09:27:15.167 INFO:teuthology.orchestra.run.smithi035.stderr: 67950K .......... .......... .......... .......... .......... 92% 281M 0s 2019-12-09T09:27:15.168 INFO:teuthology.orchestra.run.smithi035.stderr: 68000K .......... .......... .......... .......... .......... 92% 295M 0s 2019-12-09T09:27:15.168 INFO:teuthology.orchestra.run.smithi035.stderr: 68050K .......... .......... .......... .......... .......... 92% 296M 0s 2019-12-09T09:27:15.168 INFO:teuthology.orchestra.run.smithi035.stderr: 68100K .......... .......... .......... .......... .......... 92% 309M 0s 2019-12-09T09:27:15.168 INFO:teuthology.orchestra.run.smithi035.stderr: 68150K .......... .......... .......... .......... .......... 92% 280M 0s 2019-12-09T09:27:15.168 INFO:teuthology.orchestra.run.smithi035.stderr: 68200K .......... .......... .......... .......... .......... 92% 352M 0s 2019-12-09T09:27:15.168 INFO:teuthology.orchestra.run.smithi035.stderr: 68250K .......... .......... .......... .......... .......... 92% 295M 0s 2019-12-09T09:27:15.168 INFO:teuthology.orchestra.run.smithi035.stderr: 68300K .......... .......... .......... .......... .......... 92% 282M 0s 2019-12-09T09:27:15.169 INFO:teuthology.orchestra.run.smithi035.stderr: 68350K .......... .......... .......... .......... .......... 92% 270M 0s 2019-12-09T09:27:15.169 INFO:teuthology.orchestra.run.smithi035.stderr: 68400K .......... .......... .......... .......... .......... 93% 319M 0s 2019-12-09T09:27:15.169 INFO:teuthology.orchestra.run.smithi035.stderr: 68450K .......... .......... .......... .......... .......... 93% 262M 0s 2019-12-09T09:27:15.169 INFO:teuthology.orchestra.run.smithi035.stderr: 68500K .......... .......... .......... .......... .......... 93% 356M 0s 2019-12-09T09:27:15.169 INFO:teuthology.orchestra.run.smithi035.stderr: 68550K .......... .......... .......... .......... .......... 93% 266M 0s 2019-12-09T09:27:15.169 INFO:teuthology.orchestra.run.smithi035.stderr: 68600K .......... .......... .......... .......... .......... 93% 288M 0s 2019-12-09T09:27:15.169 INFO:teuthology.orchestra.run.smithi035.stderr: 68650K .......... .......... .......... .......... .......... 93% 286M 0s 2019-12-09T09:27:15.169 INFO:teuthology.orchestra.run.smithi035.stderr: 68700K .......... .......... .......... .......... .......... 93% 346M 0s 2019-12-09T09:27:15.170 INFO:teuthology.orchestra.run.smithi035.stderr: 68750K .......... .......... .......... .......... .......... 93% 299M 0s 2019-12-09T09:27:15.170 INFO:teuthology.orchestra.run.smithi035.stderr: 68800K .......... .......... .......... .......... .......... 93% 380M 0s 2019-12-09T09:27:15.170 INFO:teuthology.orchestra.run.smithi035.stderr: 68850K .......... .......... .......... .......... .......... 93% 387M 0s 2019-12-09T09:27:15.170 INFO:teuthology.orchestra.run.smithi035.stderr: 68900K .......... .......... .......... .......... .......... 93% 292M 0s 2019-12-09T09:27:15.170 INFO:teuthology.orchestra.run.smithi035.stderr: 68950K .......... .......... .......... .......... .......... 93% 262M 0s 2019-12-09T09:27:15.170 INFO:teuthology.orchestra.run.smithi035.stderr: 69000K .......... .......... .......... .......... .......... 93% 313M 0s 2019-12-09T09:27:15.170 INFO:teuthology.orchestra.run.smithi035.stderr: 69050K .......... .......... .......... .......... .......... 93% 332M 0s 2019-12-09T09:27:15.171 INFO:teuthology.orchestra.run.smithi035.stderr: 69100K .......... .......... .......... .......... .......... 94% 338M 0s 2019-12-09T09:27:15.171 INFO:teuthology.orchestra.run.smithi035.stderr: 69150K .......... .......... .......... .......... .......... 94% 292M 0s 2019-12-09T09:27:15.171 INFO:teuthology.orchestra.run.smithi035.stderr: 69200K .......... .......... .......... .......... .......... 94% 367M 0s 2019-12-09T09:27:15.171 INFO:teuthology.orchestra.run.smithi035.stderr: 69250K .......... .......... .......... .......... .......... 94% 317M 0s 2019-12-09T09:27:15.171 INFO:teuthology.orchestra.run.smithi035.stderr: 69300K .......... .......... .......... .......... .......... 94% 312M 0s 2019-12-09T09:27:15.171 INFO:teuthology.orchestra.run.smithi035.stderr: 69350K .......... .......... .......... .......... .......... 94% 339M 0s 2019-12-09T09:27:15.171 INFO:teuthology.orchestra.run.smithi035.stderr: 69400K .......... .......... .......... .......... .......... 94% 297M 0s 2019-12-09T09:27:15.172 INFO:teuthology.orchestra.run.smithi035.stderr: 69450K .......... .......... .......... .......... .......... 94% 326M 0s 2019-12-09T09:27:15.172 INFO:teuthology.orchestra.run.smithi035.stderr: 69500K .......... .......... .......... .......... .......... 94% 324M 0s 2019-12-09T09:27:15.172 INFO:teuthology.orchestra.run.smithi035.stderr: 69550K .......... .......... .......... .......... .......... 94% 313M 0s 2019-12-09T09:27:15.172 INFO:teuthology.orchestra.run.smithi035.stderr: 69600K .......... .......... .......... .......... .......... 94% 339M 0s 2019-12-09T09:27:15.172 INFO:teuthology.orchestra.run.smithi035.stderr: 69650K .......... .......... .......... .......... .......... 94% 301M 0s 2019-12-09T09:27:15.172 INFO:teuthology.orchestra.run.smithi035.stderr: 69700K .......... .......... .......... .......... .......... 94% 328M 0s 2019-12-09T09:27:15.172 INFO:teuthology.orchestra.run.smithi035.stderr: 69750K .......... .......... .......... .......... .......... 94% 318M 0s 2019-12-09T09:27:15.173 INFO:teuthology.orchestra.run.smithi035.stderr: 69800K .......... .......... .......... .......... .......... 94% 334M 0s 2019-12-09T09:27:15.173 INFO:teuthology.orchestra.run.smithi035.stderr: 69850K .......... .......... .......... .......... .......... 95% 307M 0s 2019-12-09T09:27:15.173 INFO:teuthology.orchestra.run.smithi035.stderr: 69900K .......... .......... .......... .......... .......... 95% 330M 0s 2019-12-09T09:27:15.173 INFO:teuthology.orchestra.run.smithi035.stderr: 69950K .......... .......... .......... .......... .......... 95% 259M 0s 2019-12-09T09:27:15.173 INFO:teuthology.orchestra.run.smithi035.stderr: 70000K .......... .......... .......... .......... .......... 95% 360M 0s 2019-12-09T09:27:15.173 INFO:teuthology.orchestra.run.smithi035.stderr: 70050K .......... .......... .......... .......... .......... 95% 339M 0s 2019-12-09T09:27:15.173 INFO:teuthology.orchestra.run.smithi035.stderr: 70100K .......... .......... .......... .......... .......... 95% 333M 0s 2019-12-09T09:27:15.173 INFO:teuthology.orchestra.run.smithi035.stderr: 70150K .......... .......... .......... .......... .......... 95% 292M 0s 2019-12-09T09:27:15.174 INFO:teuthology.orchestra.run.smithi035.stderr: 70200K .......... .......... .......... .......... .......... 95% 310M 0s 2019-12-09T09:27:15.174 INFO:teuthology.orchestra.run.smithi035.stderr: 70250K .......... .......... .......... .......... .......... 95% 352M 0s 2019-12-09T09:27:15.174 INFO:teuthology.orchestra.run.smithi035.stderr: 70300K .......... .......... .......... .......... .......... 95% 336M 0s 2019-12-09T09:27:15.174 INFO:teuthology.orchestra.run.smithi035.stderr: 70350K .......... .......... .......... .......... .......... 95% 269M 0s 2019-12-09T09:27:15.174 INFO:teuthology.orchestra.run.smithi035.stderr: 70400K .......... .......... .......... .......... .......... 95% 333M 0s 2019-12-09T09:27:15.174 INFO:teuthology.orchestra.run.smithi035.stderr: 70450K .......... .......... .......... .......... .......... 95% 334M 0s 2019-12-09T09:27:15.174 INFO:teuthology.orchestra.run.smithi035.stderr: 70500K .......... .......... .......... .......... .......... 95% 321M 0s 2019-12-09T09:27:15.175 INFO:teuthology.orchestra.run.smithi035.stderr: 70550K .......... .......... .......... .......... .......... 95% 290M 0s 2019-12-09T09:27:15.175 INFO:teuthology.orchestra.run.smithi035.stderr: 70600K .......... .......... .......... .......... .......... 96% 312M 0s 2019-12-09T09:27:15.175 INFO:teuthology.orchestra.run.smithi035.stderr: 70650K .......... .......... .......... .......... .......... 96% 313M 0s 2019-12-09T09:27:15.175 INFO:teuthology.orchestra.run.smithi035.stderr: 70700K .......... .......... .......... .......... .......... 96% 313M 0s 2019-12-09T09:27:15.175 INFO:teuthology.orchestra.run.smithi035.stderr: 70750K .......... .......... .......... .......... .......... 96% 283M 0s 2019-12-09T09:27:15.175 INFO:teuthology.orchestra.run.smithi035.stderr: 70800K .......... .......... .......... .......... .......... 96% 349M 0s 2019-12-09T09:27:15.175 INFO:teuthology.orchestra.run.smithi035.stderr: 70850K .......... .......... .......... .......... .......... 96% 311M 0s 2019-12-09T09:27:15.175 INFO:teuthology.orchestra.run.smithi035.stderr: 70900K .......... .......... .......... .......... .......... 96% 322M 0s 2019-12-09T09:27:15.176 INFO:teuthology.orchestra.run.smithi035.stderr: 70950K .......... .......... .......... .......... .......... 96% 270M 0s 2019-12-09T09:27:15.176 INFO:teuthology.orchestra.run.smithi035.stderr: 71000K .......... .......... .......... .......... .......... 96% 331M 0s 2019-12-09T09:27:15.176 INFO:teuthology.orchestra.run.smithi035.stderr: 71050K .......... .......... .......... .......... .......... 96% 322M 0s 2019-12-09T09:27:15.176 INFO:teuthology.orchestra.run.smithi035.stderr: 71100K .......... .......... .......... .......... .......... 96% 302M 0s 2019-12-09T09:27:15.176 INFO:teuthology.orchestra.run.smithi035.stderr: 71150K .......... .......... .......... .......... .......... 96% 252M 0s 2019-12-09T09:27:15.176 INFO:teuthology.orchestra.run.smithi035.stderr: 71200K .......... .......... .......... .......... .......... 96% 314M 0s 2019-12-09T09:27:15.176 INFO:teuthology.orchestra.run.smithi035.stderr: 71250K .......... .......... .......... .......... .......... 96% 336M 0s 2019-12-09T09:27:15.177 INFO:teuthology.orchestra.run.smithi035.stderr: 71300K .......... .......... .......... .......... .......... 96% 332M 0s 2019-12-09T09:27:15.177 INFO:teuthology.orchestra.run.smithi035.stderr: 71350K .......... .......... .......... .......... .......... 97% 242M 0s 2019-12-09T09:27:15.177 INFO:teuthology.orchestra.run.smithi035.stderr: 71400K .......... .......... .......... .......... .......... 97% 331M 0s 2019-12-09T09:27:15.177 INFO:teuthology.orchestra.run.smithi035.stderr: 71450K .......... .......... .......... .......... .......... 97% 331M 0s 2019-12-09T09:27:15.177 INFO:teuthology.orchestra.run.smithi035.stderr: 71500K .......... .......... .......... .......... .......... 97% 266M 0s 2019-12-09T09:27:15.177 INFO:teuthology.orchestra.run.smithi035.stderr: 71550K .......... .......... .......... .......... .......... 97% 256M 0s 2019-12-09T09:27:15.177 INFO:teuthology.orchestra.run.smithi035.stderr: 71600K .......... .......... .......... .......... .......... 97% 288M 0s 2019-12-09T09:27:15.178 INFO:teuthology.orchestra.run.smithi035.stderr: 71650K .......... .......... .......... .......... .......... 97% 312M 0s 2019-12-09T09:27:15.178 INFO:teuthology.orchestra.run.smithi035.stderr: 71700K .......... .......... .......... .......... .......... 97% 329M 0s 2019-12-09T09:27:15.178 INFO:teuthology.orchestra.run.smithi035.stderr: 71750K .......... .......... .......... .......... .......... 97% 263M 0s 2019-12-09T09:27:15.178 INFO:teuthology.orchestra.run.smithi035.stderr: 71800K .......... .......... .......... .......... .......... 97% 320M 0s 2019-12-09T09:27:15.178 INFO:teuthology.orchestra.run.smithi035.stderr: 71850K .......... .......... .......... .......... .......... 97% 333M 0s 2019-12-09T09:27:15.178 INFO:teuthology.orchestra.run.smithi035.stderr: 71900K .......... .......... .......... .......... .......... 97% 305M 0s 2019-12-09T09:27:15.178 INFO:teuthology.orchestra.run.smithi035.stderr: 71950K .......... .......... .......... .......... .......... 97% 309M 0s 2019-12-09T09:27:15.179 INFO:teuthology.orchestra.run.smithi035.stderr: 72000K .......... .......... .......... .......... .......... 97% 318M 0s 2019-12-09T09:27:15.179 INFO:teuthology.orchestra.run.smithi035.stderr: 72050K .......... .......... .......... .......... .......... 98% 329M 0s 2019-12-09T09:27:15.179 INFO:teuthology.orchestra.run.smithi035.stderr: 72100K .......... .......... .......... .......... .......... 98% 366M 0s 2019-12-09T09:27:15.179 INFO:teuthology.orchestra.run.smithi035.stderr: 72150K .......... .......... .......... .......... .......... 98% 291M 0s 2019-12-09T09:27:15.179 INFO:teuthology.orchestra.run.smithi035.stderr: 72200K .......... .......... .......... .......... .......... 98% 336M 0s 2019-12-09T09:27:15.179 INFO:teuthology.orchestra.run.smithi035.stderr: 72250K .......... .......... .......... .......... .......... 98% 327M 0s 2019-12-09T09:27:15.179 INFO:teuthology.orchestra.run.smithi035.stderr: 72300K .......... .......... .......... .......... .......... 98% 296M 0s 2019-12-09T09:27:15.179 INFO:teuthology.orchestra.run.smithi035.stderr: 72350K .......... .......... .......... .......... .......... 98% 301M 0s 2019-12-09T09:27:15.180 INFO:teuthology.orchestra.run.smithi035.stderr: 72400K .......... .......... .......... .......... .......... 98% 273M 0s 2019-12-09T09:27:15.180 INFO:teuthology.orchestra.run.smithi035.stderr: 72450K .......... .......... .......... .......... .......... 98% 301M 0s 2019-12-09T09:27:15.180 INFO:teuthology.orchestra.run.smithi035.stderr: 72500K .......... .......... .......... .......... .......... 98% 301M 0s 2019-12-09T09:27:15.180 INFO:teuthology.orchestra.run.smithi035.stderr: 72550K .......... .......... .......... .......... .......... 98% 259M 0s 2019-12-09T09:27:15.180 INFO:teuthology.orchestra.run.smithi035.stderr: 72600K .......... .......... .......... .......... .......... 98% 314M 0s 2019-12-09T09:27:15.180 INFO:teuthology.orchestra.run.smithi035.stderr: 72650K .......... .......... .......... .......... .......... 98% 313M 0s 2019-12-09T09:27:15.180 INFO:teuthology.orchestra.run.smithi035.stderr: 72700K .......... .......... .......... .......... .......... 98% 391M 0s 2019-12-09T09:27:15.181 INFO:teuthology.orchestra.run.smithi035.stderr: 72750K .......... .......... .......... .......... .......... 98% 319M 0s 2019-12-09T09:27:15.181 INFO:teuthology.orchestra.run.smithi035.stderr: 72800K .......... .......... .......... .......... .......... 99% 396M 0s 2019-12-09T09:27:15.181 INFO:teuthology.orchestra.run.smithi035.stderr: 72850K .......... .......... .......... .......... .......... 99% 393M 0s 2019-12-09T09:27:15.181 INFO:teuthology.orchestra.run.smithi035.stderr: 72900K .......... .......... .......... .......... .......... 99% 390M 0s 2019-12-09T09:27:15.181 INFO:teuthology.orchestra.run.smithi035.stderr: 72950K .......... .......... .......... .......... .......... 99% 345M 0s 2019-12-09T09:27:15.181 INFO:teuthology.orchestra.run.smithi035.stderr: 73000K .......... .......... .......... .......... .......... 99% 379M 0s 2019-12-09T09:27:15.181 INFO:teuthology.orchestra.run.smithi035.stderr: 73050K .......... .......... .......... .......... .......... 99% 388M 0s 2019-12-09T09:27:15.182 INFO:teuthology.orchestra.run.smithi035.stderr: 73100K .......... .......... .......... .......... .......... 99% 352M 0s 2019-12-09T09:27:15.182 INFO:teuthology.orchestra.run.smithi035.stderr: 73150K .......... .......... .......... .......... .......... 99% 306M 0s 2019-12-09T09:27:15.182 INFO:teuthology.orchestra.run.smithi035.stderr: 73200K .......... .......... .......... .......... .......... 99% 378M 0s 2019-12-09T09:27:15.182 INFO:teuthology.orchestra.run.smithi035.stderr: 73250K .......... .......... .......... .......... .......... 99% 373M 0s 2019-12-09T09:27:15.182 INFO:teuthology.orchestra.run.smithi035.stderr: 73300K .......... .......... .......... .......... .......... 99% 375M 0s 2019-12-09T09:27:15.182 INFO:teuthology.orchestra.run.smithi035.stderr: 73350K .......... .......... .......... .......... .......... 99% 338M 0s 2019-12-09T09:27:15.182 INFO:teuthology.orchestra.run.smithi035.stderr: 73400K .......... .......... .......... .......... .......... 99% 378M 0s 2019-12-09T09:27:15.182 INFO:teuthology.orchestra.run.smithi035.stderr: 73450K .......... .......... .......... .......... .......... 99% 368M 0s 2019-12-09T09:27:15.183 INFO:teuthology.orchestra.run.smithi035.stderr: 73500K .......... .......... .......... .......... .......... 99% 364M 0s 2019-12-09T09:27:15.183 INFO:teuthology.orchestra.run.smithi035.stderr: 73550K ........ 100% 206M=0.4s 2019-12-09T09:27:15.183 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T09:27:15.183 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09 09:27:14 (203 MB/s) - ‘gradle-4.7-bin.zip’ saved [75323957/75323957] 2019-12-09T09:27:15.183 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T09:27:15.183 INFO:teuthology.orchestra.run.smithi035.stdout:Archive: gradle-4.7-bin.zip 2019-12-09T09:27:15.184 INFO:teuthology.orchestra.run.smithi035.stdout: creating: /opt/gradle/gradle-4.7/ 2019-12-09T09:27:15.184 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/getting-started.html 2019-12-09T09:27:15.184 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/LICENSE 2019-12-09T09:27:15.184 INFO:teuthology.orchestra.run.smithi035.stdout: creating: /opt/gradle/gradle-4.7/media/ 2019-12-09T09:27:15.184 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-64x64.png 2019-12-09T09:27:15.184 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-48x48.png 2019-12-09T09:27:15.184 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle.icns 2019-12-09T09:27:15.185 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-24x24.png 2019-12-09T09:27:15.185 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-256x256.png 2019-12-09T09:27:15.185 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-512x512.png 2019-12-09T09:27:15.185 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-16x16.png 2019-12-09T09:27:15.185 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-32x32.png 2019-12-09T09:27:15.185 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-128x128.png 2019-12-09T09:27:15.185 INFO:teuthology.orchestra.run.smithi035.stdout: creating: /opt/gradle/gradle-4.7/init.d/ 2019-12-09T09:27:15.185 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/init.d/readme.txt 2019-12-09T09:27:15.186 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/NOTICE 2019-12-09T09:27:15.186 INFO:teuthology.orchestra.run.smithi035.stdout: creating: /opt/gradle/gradle-4.7/bin/ 2019-12-09T09:27:15.186 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle.bat 2019-12-09T09:27:15.186 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle 2019-12-09T09:27:15.186 INFO:teuthology.orchestra.run.smithi035.stdout: creating: /opt/gradle/gradle-4.7/lib/ 2019-12-09T09:27:15.186 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/annotations-13.0.jar 2019-12-09T09:27:15.186 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-0.16.3.jar 2019-12-09T09:27:15.187 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-builders-0.16.3.jar 2019-12-09T09:27:15.187 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-models-0.16.3.jar 2019-12-09T09:27:15.187 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-compiler-embeddable-1.2.31.jar 2019-12-09T09:27:15.187 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-reflect-1.2.31.jar 2019-12-09T09:27:15.187 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-sam-with-receiver-compiler-plugin-1.2.31.jar 2019-12-09T09:27:15.187 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-script-runtime-1.2.31.jar 2019-12-09T09:27:15.187 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-1.2.31.jar 2019-12-09T09:27:15.188 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk7-1.2.31.jar 2019-12-09T09:27:15.188 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk8-1.2.31.jar 2019-12-09T09:27:15.188 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-launcher-4.7.jar 2019-12-09T09:27:15.188 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-runtime-api-info-4.7.jar 2019-12-09T09:27:15.188 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-tooling-api-4.7.jar 2019-12-09T09:27:15.188 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-wrapper-4.7.jar 2019-12-09T09:27:15.188 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-installation-beacon-4.7.jar 2019-12-09T09:27:15.189 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-4.7.jar 2019-12-09T09:27:15.189 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-jvm-services-4.7.jar 2019-12-09T09:27:15.189 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-api-4.7.jar 2019-12-09T09:27:15.189 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-cache-4.7.jar 2019-12-09T09:27:15.189 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-logging-4.7.jar 2019-12-09T09:27:15.189 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-process-services-4.7.jar 2019-12-09T09:27:15.189 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-resources-4.7.jar 2019-12-09T09:27:15.189 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-persistent-cache-4.7.jar 2019-12-09T09:27:15.190 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-native-4.7.jar 2019-12-09T09:27:15.190 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-groovy-4.7.jar 2019-12-09T09:27:15.190 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-core-4.7.jar 2019-12-09T09:27:15.190 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-groovy-4.7.jar 2019-12-09T09:27:15.190 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-messaging-4.7.jar 2019-12-09T09:27:15.190 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-4.7.jar 2019-12-09T09:27:15.190 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-option-4.7.jar 2019-12-09T09:27:15.191 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-cli-4.7.jar 2019-12-09T09:27:15.191 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-docs-4.7.jar 2019-12-09T09:27:15.191 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-commons-6.1.1.jar 2019-12-09T09:27:15.191 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-analysis-6.1.1.jar 2019-12-09T09:27:15.191 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-tree-6.1.1.jar 2019-12-09T09:27:15.191 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/kryo-2.20.jar 2019-12-09T09:27:15.191 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/reflectasm-1.07-shaded.jar 2019-12-09T09:27:15.192 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-6.1.1.jar 2019-12-09T09:27:15.192 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-io-2.4.jar 2019-12-09T09:27:15.192 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcl-over-slf4j-1.7.16.jar 2019-12-09T09:27:15.192 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/jul-to-slf4j-1.7.16.jar 2019-12-09T09:27:15.192 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/log4j-over-slf4j-1.7.16.jar 2019-12-09T09:27:15.192 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/slf4j-api-1.7.16.jar 2019-12-09T09:27:15.192 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/guava-jdk5-17.0.jar 2019-12-09T09:27:15.193 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/jsr305-2.0.1.jar 2019-12-09T09:27:15.193 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/fastutil-7.2.1-min.jar 2019-12-09T09:27:15.193 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-lang-2.6.jar 2019-12-09T09:27:15.193 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcip-annotations-1.0.jar 2019-12-09T09:27:15.193 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/groovy-all-2.4.12.jar 2019-12-09T09:27:15.193 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-1.9.9.jar 2019-12-09T09:27:15.194 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/javax.inject-1.jar 2019-12-09T09:27:15.194 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-collections-3.2.2.jar 2019-12-09T09:27:15.194 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-0.14.jar 2019-12-09T09:27:15.194 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-compress-1.14.jar 2019-12-09T09:27:15.194 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/xml-apis-1.4.01.jar 2019-12-09T09:27:15.194 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/jansi-1.14.jar 2019-12-09T09:27:15.194 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-launcher-1.9.9.jar 2019-12-09T09:27:15.195 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libcpp-0.14.jar 2019-12-09T09:27:15.195 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libstdcpp-0.14.jar 2019-12-09T09:27:15.195 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libcpp-0.14.jar 2019-12-09T09:27:15.195 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libstdcpp-0.14.jar 2019-12-09T09:27:15.195 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-0.14.jar 2019-12-09T09:27:15.195 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-0.14.jar 2019-12-09T09:27:15.195 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-amd64-0.14.jar 2019-12-09T09:27:15.196 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-i386-0.14.jar 2019-12-09T09:27:15.196 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-amd64-0.14.jar 2019-12-09T09:27:15.196 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-i386-0.14.jar 2019-12-09T09:27:15.196 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses5-0.14.jar 2019-12-09T09:27:15.196 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses6-0.14.jar 2019-12-09T09:27:15.196 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses5-0.14.jar 2019-12-09T09:27:15.196 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses6-0.14.jar 2019-12-09T09:27:15.197 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/minlog-1.2.jar 2019-12-09T09:27:15.197 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/objenesis-1.2.jar 2019-12-09T09:27:15.197 INFO:teuthology.orchestra.run.smithi035.stdout: creating: /opt/gradle/gradle-4.7/lib/plugins/ 2019-12-09T09:27:15.197 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-announce-4.7.jar 2019-12-09T09:27:15.197 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-antlr-4.7.jar 2019-12-09T09:27:15.197 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-code-quality-4.7.jar 2019-12-09T09:27:15.197 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-osgi-4.7.jar 2019-12-09T09:27:15.197 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-signing-4.7.jar 2019-12-09T09:27:15.198 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-native-4.7.jar 2019-12-09T09:27:15.198 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-native-4.7.jar 2019-12-09T09:27:15.198 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-native-4.7.jar 2019-12-09T09:27:15.198 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-development-4.7.jar 2019-12-09T09:27:15.198 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-maven-4.7.jar 2019-12-09T09:27:15.198 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-play-4.7.jar 2019-12-09T09:27:15.198 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-4.7.jar 2019-12-09T09:27:15.199 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-scala-4.7.jar 2019-12-09T09:27:15.199 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-comparison-4.7.jar 2019-12-09T09:27:15.199 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ear-4.7.jar 2019-12-09T09:27:15.199 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-play-4.7.jar 2019-12-09T09:27:15.199 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-javascript-4.7.jar 2019-12-09T09:27:15.199 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ivy-4.7.jar 2019-12-09T09:27:15.199 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-jacoco-4.7.jar 2019-12-09T09:27:15.200 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-init-4.7.jar 2019-12-09T09:27:15.200 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugins-4.7.jar 2019-12-09T09:27:15.200 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-tooling-api-builders-4.7.jar 2019-12-09T09:27:15.200 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-test-kit-4.7.jar 2019-12-09T09:27:15.200 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-composite-builds-4.7.jar 2019-12-09T09:27:15.200 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-groovy-4.7.jar 2019-12-09T09:27:15.200 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-scala-4.7.jar 2019-12-09T09:27:15.201 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-junit-platform-4.7.jar 2019-12-09T09:27:15.201 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-jvm-4.7.jar 2019-12-09T09:27:15.201 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-java-4.7.jar 2019-12-09T09:27:15.201 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-jvm-4.7.jar 2019-12-09T09:27:15.201 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-jvm-4.7.jar 2019-12-09T09:27:15.201 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-native-4.7.jar 2019-12-09T09:27:15.201 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-diagnostics-4.7.jar 2019-12-09T09:27:15.202 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-base-4.7.jar 2019-12-09T09:27:15.202 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-reporting-4.7.jar 2019-12-09T09:27:15.202 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-publish-4.7.jar 2019-12-09T09:27:15.202 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-base-4.7.jar 2019-12-09T09:27:15.202 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-use-4.7.jar 2019-12-09T09:27:15.202 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-s3-4.7.jar 2019-12-09T09:27:15.202 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-gcs-4.7.jar 2019-12-09T09:27:15.202 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-cache-http-4.7.jar 2019-12-09T09:27:15.203 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-dependency-management-4.7.jar 2019-12-09T09:27:15.203 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-http-4.7.jar 2019-12-09T09:27:15.203 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-sftp-4.7.jar 2019-12-09T09:27:15.203 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-workers-4.7.jar 2019-12-09T09:27:15.203 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-version-control-4.7.jar 2019-12-09T09:27:15.203 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-s3-1.11.267.jar 2019-12-09T09:27:15.203 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-kms-1.11.267.jar 2019-12-09T09:27:15.204 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-core-1.11.267.jar 2019-12-09T09:27:15.204 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/org.eclipse.jgit-4.5.3.201708160445-r.jar 2019-12-09T09:27:15.204 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-2.4.jar 2019-12-09T09:27:15.204 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-shared4-2.4.jar 2019-12-09T09:27:15.204 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-services-storage-v1-rev116-1.23.0.jar 2019-12-09T09:27:15.204 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-client-1.23.0.jar 2019-12-09T09:27:15.204 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-oauth-client-1.23.0.jar 2019-12-09T09:27:15.205 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-jackson2-1.23.0.jar 2019-12-09T09:27:15.205 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-1.23.0.jar 2019-12-09T09:27:15.205 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpclient-4.5.5.jar 2019-12-09T09:27:15.205 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-codec-1.10.jar 2019-12-09T09:27:15.205 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-container-default-1.5.5.jar 2019-12-09T09:27:15.205 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xbean-reflect-3.4.jar 2019-12-09T09:27:15.205 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/asm-util-6.1.1.jar 2019-12-09T09:27:15.206 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jatl-0.2.2.jar 2019-12-09T09:27:15.206 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-cli-1.2.jar 2019-12-09T09:27:15.206 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/biz.aQute.bndlib-3.4.0.jar 2019-12-09T09:27:15.206 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ivy-2.2.0.jar 2019-12-09T09:27:15.206 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-compat-3.0.4.jar 2019-12-09T09:27:15.206 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-core-3.0.4.jar 2019-12-09T09:27:15.206 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-common-0.8-20100325.jar 2019-12-09T09:27:15.206 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-groovy-0.8-20100325.jar 2019-12-09T09:27:15.207 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-file-2.4.jar 2019-12-09T09:27:15.207 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-connector-wagon-1.13.1.jar 2019-12-09T09:27:15.207 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcpg-jdk15on-1.58.jar 2019-12-09T09:27:15.207 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/rhino-1.7R3.jar 2019-12-09T09:27:15.207 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gson-2.7.jar 2019-12-09T09:27:15.207 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/simple-4.1.21.jar 2019-12-09T09:27:15.207 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-plugin-api-3.0.4.jar 2019-12-09T09:27:15.208 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/testng-6.3.1.jar 2019-12-09T09:27:15.208 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/snakeyaml-1.6.jar 2019-12-09T09:27:15.208 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/dd-plist-1.20.jar 2019-12-09T09:27:15.208 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-4.12.jar 2019-12-09T09:27:15.208 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bsh-2.0b6.jar 2019-12-09T09:27:15.208 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-launcher-1.0.3.jar 2019-12-09T09:27:15.208 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcifs-1.3.17.jar 2019-12-09T09:27:15.209 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/nekohtml-1.9.20.jar 2019-12-09T09:27:15.209 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xercesImpl-2.11.0.jar 2019-12-09T09:27:15.209 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jsch-0.1.54.jar 2019-12-09T09:27:15.209 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jmespath-java-1.11.267.jar 2019-12-09T09:27:15.209 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-databind-2.8.11.jar 2019-12-09T09:27:15.209 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-core-2.8.11.jar 2019-12-09T09:27:15.209 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-annotations-2.8.11.jar 2019-12-09T09:27:15.209 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/joda-time-2.8.2.jar 2019-12-09T09:27:15.210 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcprov-jdk15on-1.58.jar 2019-12-09T09:27:15.210 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-aether-provider-3.0.4.jar 2019-12-09T09:27:15.210 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-impl-1.13.1.jar 2019-12-09T09:27:15.210 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-spi-1.13.1.jar 2019-12-09T09:27:15.210 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-util-1.13.1.jar 2019-12-09T09:27:15.210 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-api-1.13.1.jar 2019-12-09T09:27:15.211 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-provider-api-2.4.jar 2019-12-09T09:27:15.211 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-classworlds-2.4.jar 2019-12-09T09:27:15.211 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-builder-3.0.4.jar 2019-12-09T09:27:15.211 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-3.0.4.jar 2019-12-09T09:27:15.211 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-builder-3.0.4.jar 2019-12-09T09:27:15.211 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-artifact-3.0.4.jar 2019-12-09T09:27:15.211 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-3.0.4.jar 2019-12-09T09:27:15.212 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-sec-dispatcher-1.3.jar 2019-12-09T09:27:15.212 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-repository-metadata-3.0.4.jar 2019-12-09T09:27:15.212 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-utils-3.0.8.jar 2019-12-09T09:27:15.212 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ion-java-1.0.2.jar 2019-12-09T09:27:15.212 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpcore-4.4.9.jar 2019-12-09T09:27:15.212 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/hamcrest-core-1.3.jar 2019-12-09T09:27:15.213 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-engine-1.0.3.jar 2019-12-09T09:27:15.213 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-commons-1.0.3.jar 2019-12-09T09:27:15.213 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/apiguardian-api-1.0.0.jar 2019-12-09T09:27:15.213 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcommander-1.47.jar 2019-12-09T09:27:15.213 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-interpolation-1.14.jar 2019-12-09T09:27:15.213 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-component-annotations-1.5.5.jar 2019-12-09T09:27:15.213 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/opentest4j-1.0.0.jar 2019-12-09T09:27:15.214 INFO:teuthology.orchestra.run.smithi035.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-cipher-1.7.jar 2019-12-09T09:27:15.214 INFO:teuthology.orchestra.run.smithi035.stdout:export PATH=/opt/gradle/gradle-4.7/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin 2019-12-09T09:27:16.229 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:27:16.229 INFO:teuthology.orchestra.run.smithi035.stdout:Welcome to Gradle 4.7! 2019-12-09T09:27:16.229 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:27:16.230 INFO:teuthology.orchestra.run.smithi035.stdout:Here are the highlights of this release: 2019-12-09T09:27:16.230 INFO:teuthology.orchestra.run.smithi035.stdout: - Incremental annotation processing 2019-12-09T09:27:16.231 INFO:teuthology.orchestra.run.smithi035.stdout: - JDK 10 support 2019-12-09T09:27:16.231 INFO:teuthology.orchestra.run.smithi035.stdout: - Grouped non-interactive console logs 2019-12-09T09:27:16.231 INFO:teuthology.orchestra.run.smithi035.stdout: - Failed tests are re-run first for quicker feedback 2019-12-09T09:27:16.231 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:27:16.231 INFO:teuthology.orchestra.run.smithi035.stdout:For more details see https://docs.gradle.org/4.7/release-notes.html 2019-12-09T09:27:16.232 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:27:16.525 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:27:16.526 INFO:teuthology.orchestra.run.smithi035.stdout:------------------------------------------------------------ 2019-12-09T09:27:16.526 INFO:teuthology.orchestra.run.smithi035.stdout:Gradle 4.7 2019-12-09T09:27:16.526 INFO:teuthology.orchestra.run.smithi035.stdout:------------------------------------------------------------ 2019-12-09T09:27:16.526 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:27:16.526 INFO:teuthology.orchestra.run.smithi035.stdout:Build time: 2018-04-18 09:09:12 UTC 2019-12-09T09:27:16.526 INFO:teuthology.orchestra.run.smithi035.stdout:Revision: b9a962bf70638332300e7f810689cb2febbd4a6c 2019-12-09T09:27:16.526 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:27:16.527 INFO:teuthology.orchestra.run.smithi035.stdout:Groovy: 2.4.12 2019-12-09T09:27:16.527 INFO:teuthology.orchestra.run.smithi035.stdout:Ant: Apache Ant(TM) version 1.9.9 compiled on February 2 2017 2019-12-09T09:27:16.527 INFO:teuthology.orchestra.run.smithi035.stdout:JVM: 1.8.0_232 (Oracle Corporation 25.232-b09) 2019-12-09T09:27:16.527 INFO:teuthology.orchestra.run.smithi035.stdout:OS: Linux 3.10.0-1062.9.1.el7.x86_64 amd64 2019-12-09T09:27:16.527 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:27:16.531 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2019-12-09T09:27:16.532 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-09T09:27:16.532 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi035.front.sepia.ceph.com 2019-12-09T09:27:16.532 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-12-09T09:27:16.532 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi022.front.sepia.ceph.com 2019-12-09T09:27:16.532 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2019-12-09T09:27:16.947 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 's3dir': 'folder', 'endpoint': None, 'display_name': None, 'region': 'mexico', 'port': None, 'access_secret': None, 'SSE': 'AES256', 'is_secure': None, 'email': None, 'dir': '../data'}} 2019-12-09T09:27:16.980 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2019-12-09T09:27:16.980 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'IZUMPJOIOIZXSABFSTRP', 's3dir': 'folder', 'endpoint': 'smithi035.front.sepia.ceph.com:80', 'display_name': 'Ms. tester.client.0', 'user_id': 'tester.client.0', 'region': 'mexico', 'port': 80, 'access_secret': 'CqmlT4lRl5LvuFNCztFZEbBqlU0sKZfORyBaKIYqMpTViXcy00WecA==', 'host': 'smithi035.front.sepia.ceph.com', 'SSE': 'AES256', 'is_secure': False, 'email': 'tester.client.0_test@test.test', 'dir': '../data'} 2019-12-09T09:27:16.980 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2019-12-09T09:27:16.981 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'IZUMPJOIOIZXSABFSTRP', '--secret', 'CqmlT4lRl5LvuFNCztFZEbBqlU0sKZfORyBaKIYqMpTViXcy00WecA==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2019-12-09T09:27:16.982 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key IZUMPJOIOIZXSABFSTRP --secret CqmlT4lRl5LvuFNCztFZEbBqlU0sKZfORyBaKIYqMpTViXcy00WecA== --email tester.client.0_test@test.test --cluster ceph 2019-12-09T09:27:17.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.042+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.141 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.072+0000 7f1c8d99bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:17.141 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.072+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.152 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.083+0000 7f1c8d99bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:17.153 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.083+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.158 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.089+0000 7f1c8d99bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:17.158 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.089+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.105+0000 7f1c8d99bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:17.375 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.306+0000 7f1c8d99bec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:27:17.375 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.306+0000 7f1c8d99bec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:27:17.375 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.306+0000 7f1c8d99bec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:27:17.375 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.306+0000 7f1c8d99bec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:27:17.375 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.306+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.314+0000 7f1c8d99bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:17.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.314+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.396 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.327+0000 7f1c8d99bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:27:17.396 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.327+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.404 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.335+0000 7f1c8d99bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:17.404 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.335+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.416 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.347+0000 7f1c8d99bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:27:17.416 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.347+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.353+0000 7f1c8d99bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:17.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.353+0000 7f1c8d99bec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:27:17.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.353+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.428 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.359+0000 7f1c8d99bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:17.428 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.359+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.438 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.370+0000 7f1c8d99bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:17.439 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.370+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.454 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.385+0000 7f1c8d99bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:27:17.454 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.385+0000 7f1c8d99bec0 20 zonegroup default 2019-12-09T09:27:17.455 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.385+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.392+0000 7f1c8d99bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:17.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.392+0000 7f1c8d99bec0 10 Cannot find current period zone using local zone 2019-12-09T09:27:17.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.392+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.396+0000 7f1c8d99bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:17.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.396+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.473 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.404+0000 7f1c8d99bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:17.473 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.404+0000 7f1c8d99bec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:17.480 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.411+0000 7f1c8d99bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:27:17.480 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.411+0000 7f1c8d99bec0 20 zone default found 2019-12-09T09:27:17.480 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.411+0000 7f1c8d99bec0 20 started sync module instance, tier type = 2019-12-09T09:27:17.481 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.411+0000 7f1c8d99bec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:27:17.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.732+0000 7f1c8d99bec0 20 add_watcher() i=0 2019-12-09T09:27:17.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.732+0000 7f1c8d99bec0 20 add_watcher() i=1 2019-12-09T09:27:17.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.732+0000 7f1c8d99bec0 20 add_watcher() i=2 2019-12-09T09:27:17.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.732+0000 7f1c8d99bec0 20 add_watcher() i=3 2019-12-09T09:27:17.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.732+0000 7f1c8d99bec0 20 add_watcher() i=4 2019-12-09T09:27:17.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.732+0000 7f1c8d99bec0 20 add_watcher() i=5 2019-12-09T09:27:17.802 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.732+0000 7f1c8d99bec0 20 add_watcher() i=6 2019-12-09T09:27:17.831 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.762+0000 7f1c8d99bec0 20 add_watcher() i=7 2019-12-09T09:27:17.832 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.762+0000 7f1c8d99bec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:27:17.832 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.763+0000 7f1c4b7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:27:17.837 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.768+0000 7f1c7b3c6700 0 -- 172.21.15.35:0/2494959674 >> v2:172.21.15.22:3300/0 conn(0x560319daef60 msgr2=0x560319daf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T09:27:18.040 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:17.971+0000 7f1c7b3c6700 0 --2- 172.21.15.35:0/2494959674 >> v2:172.21.15.22:3300/0 conn(0x560319daef60 0x560319daf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T09:27:18.109 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:18.040+0000 7f1c7abc5700 0 -- 172.21.15.35:0/1723899238 >> [v2:172.21.15.22:6800/8484,v1:172.21.15.22:6801/8484] conn(0x7f1bbc0224a0 msgr2=0x7f1bbc024990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:27:19.713 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.640+0000 7f1bb9ffb700 20 reqs_thread_entry: start 2019-12-09T09:27:19.713 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.641+0000 7f1bb37fe700 20 reqs_thread_entry: start 2019-12-09T09:27:19.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.641+0000 7f1c8d99bec0 20 init_complete bucket index max shards: 0 2019-12-09T09:27:19.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.641+0000 7f1bb1ffb700 20 reqs_thread_entry: start 2019-12-09T09:27:19.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.641+0000 7f1c8d99bec0 20 get_system_obj_state: rctx=0x7ffe6c7ce3c8 obj=default.rgw.meta:users.uid:tester.client.0 state=0x560319e34150 s->prefetch_data=0 2019-12-09T09:27:19.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.641+0000 7f1c8d99bec0 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2019-12-09T09:27:19.720 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.651+0000 7f1c8d99bec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2019-12-09T09:27:19.720 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.651+0000 7f1c8d99bec0 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T09:27:19.720 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.651+0000 7f1c8d99bec0 20 get_system_obj_state: rctx=0x7ffe6c7ce3d8 obj=default.rgw.meta:users.email:tester.client.0_test@test.test state=0x560319e35ee0 s->prefetch_data=0 2019-12-09T09:27:19.721 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.651+0000 7f1c8d99bec0 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2019-12-09T09:27:19.728 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.659+0000 7f1c8d99bec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2019-12-09T09:27:19.729 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.659+0000 7f1c8d99bec0 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T09:27:19.729 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.659+0000 7f1c8d99bec0 20 get_system_obj_state: rctx=0x7ffe6c7ce3d8 obj=default.rgw.meta:users.keys:IZUMPJOIOIZXSABFSTRP state=0x560319e35ee0 s->prefetch_data=0 2019-12-09T09:27:19.729 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.659+0000 7f1c8d99bec0 10 cache get: name=default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP : miss 2019-12-09T09:27:19.737 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.668+0000 7f1c8d99bec0 10 cache put: name=default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP info.flags=0x0 2019-12-09T09:27:19.737 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.668+0000 7f1c8d99bec0 10 adding default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP to cache LRU end 2019-12-09T09:27:19.737 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.668+0000 7f1c8d99bec0 20 get_system_obj_state: rctx=0x7ffe6c7cdfe8 obj=default.rgw.meta:users.keys:IZUMPJOIOIZXSABFSTRP state=0x560319e35ee0 s->prefetch_data=0 2019-12-09T09:27:19.737 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.668+0000 7f1c8d99bec0 10 cache get: name=default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP : type miss (requested=0x6, cached=0x0) 2019-12-09T09:27:19.745 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.676+0000 7f1c8d99bec0 10 cache put: name=default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP info.flags=0x0 2019-12-09T09:27:19.745 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.676+0000 7f1c8d99bec0 10 moving default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP to cache LRU end 2019-12-09T09:27:19.745 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.676+0000 7f1c8d99bec0 20 get_system_obj_state: rctx=0x7ffe6c7ce168 obj=default.rgw.meta:users.keys:IZUMPJOIOIZXSABFSTRP state=0x560319e35ee0 s->prefetch_data=0 2019-12-09T09:27:19.746 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.676+0000 7f1c8d99bec0 10 cache get: name=default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP : type miss (requested=0x6, cached=0x0) 2019-12-09T09:27:19.753 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.684+0000 7f1c8d99bec0 10 cache put: name=default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP info.flags=0x0 2019-12-09T09:27:19.753 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.684+0000 7f1c8d99bec0 10 moving default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP to cache LRU end 2019-12-09T09:27:19.809 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.740+0000 7f1c8d99bec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-12-09T09:27:19.809 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.740+0000 7f1c8d99bec0 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T09:27:19.809 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.740+0000 7f1c8d99bec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=632 2019-12-09T09:27:19.826 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.757+0000 7f1c5bfff700 10 RGWWatcher::handle_notify() notify_id 575525617669 cookie 94571318603520 notifier 4508 bl.length()=632 2019-12-09T09:27:19.826 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.757+0000 7f1c5bfff700 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-12-09T09:27:19.827 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:19.757+0000 7f1c5bfff700 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T09:27:20.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.017+0000 7f1c8d99bec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-12-09T09:27:20.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.017+0000 7f1c8d99bec0 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T09:27:20.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.017+0000 7f1c8d99bec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=200 2019-12-09T09:27:20.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.028+0000 7f1c5bfff700 10 RGWWatcher::handle_notify() notify_id 575525617668 cookie 94571318584912 notifier 4508 bl.length()=200 2019-12-09T09:27:20.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.028+0000 7f1c5bfff700 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-12-09T09:27:20.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.028+0000 7f1c5bfff700 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T09:27:20.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.246+0000 7f1c8d99bec0 10 cache put: name=default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP info.flags=0x7 2019-12-09T09:27:20.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.246+0000 7f1c8d99bec0 10 moving default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP to cache LRU end 2019-12-09T09:27:20.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.246+0000 7f1c8d99bec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=189 2019-12-09T09:27:20.329 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.260+0000 7f1c5bfff700 10 RGWWatcher::handle_notify() notify_id 575525617670 cookie 94571318618976 notifier 4508 bl.length()=189 2019-12-09T09:27:20.329 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.260+0000 7f1c5bfff700 10 cache put: name=default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP info.flags=0x7 2019-12-09T09:27:20.329 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.260+0000 7f1c5bfff700 10 moving default.rgw.meta+users.keys+IZUMPJOIOIZXSABFSTRP to cache LRU end 2019-12-09T09:27:20.388 INFO:teuthology.orchestra.run.smithi035.stdout:{ 2019-12-09T09:27:20.388 INFO:teuthology.orchestra.run.smithi035.stdout: "user_id": "tester.client.0", 2019-12-09T09:27:20.388 INFO:teuthology.orchestra.run.smithi035.stdout: "display_name": "Ms. tester.client.0", 2019-12-09T09:27:20.388 INFO:teuthology.orchestra.run.smithi035.stdout: "email": "tester.client.0_test@test.test", 2019-12-09T09:27:20.388 INFO:teuthology.orchestra.run.smithi035.stdout: "suspended": 0, 2019-12-09T09:27:20.389 INFO:teuthology.orchestra.run.smithi035.stdout: "max_buckets": 1000, 2019-12-09T09:27:20.389 INFO:teuthology.orchestra.run.smithi035.stdout: "subusers": [], 2019-12-09T09:27:20.389 INFO:teuthology.orchestra.run.smithi035.stdout: "keys": [ 2019-12-09T09:27:20.389 INFO:teuthology.orchestra.run.smithi035.stdout: { 2019-12-09T09:27:20.389 INFO:teuthology.orchestra.run.smithi035.stdout: "user": "tester.client.0", 2019-12-09T09:27:20.389 INFO:teuthology.orchestra.run.smithi035.stdout: "access_key": "IZUMPJOIOIZXSABFSTRP", 2019-12-09T09:27:20.389 INFO:teuthology.orchestra.run.smithi035.stdout: "secret_key": "CqmlT4lRl5LvuFNCztFZEbBqlU0sKZfORyBaKIYqMpTViXcy00WecA==" 2019-12-09T09:27:20.390 INFO:teuthology.orchestra.run.smithi035.stdout: } 2019-12-09T09:27:20.390 INFO:teuthology.orchestra.run.smithi035.stdout: ], 2019-12-09T09:27:20.390 INFO:teuthology.orchestra.run.smithi035.stdout: "swift_keys": [], 2019-12-09T09:27:20.390 INFO:teuthology.orchestra.run.smithi035.stdout: "caps": [], 2019-12-09T09:27:20.390 INFO:teuthology.orchestra.run.smithi035.stdout: "op_mask": "read, write, delete", 2019-12-09T09:27:20.390 INFO:teuthology.orchestra.run.smithi035.stdout: "default_placement": "", 2019-12-09T09:27:20.390 INFO:teuthology.orchestra.run.smithi035.stdout: "default_storage_class": "", 2019-12-09T09:27:20.391 INFO:teuthology.orchestra.run.smithi035.stdout: "placement_tags": [], 2019-12-09T09:27:20.391 INFO:teuthology.orchestra.run.smithi035.stdout: "bucket_quota": { 2019-12-09T09:27:20.391 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T09:27:20.391 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T09:27:20.391 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T09:27:20.391 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T09:27:20.391 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T09:27:20.392 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T09:27:20.392 INFO:teuthology.orchestra.run.smithi035.stdout: "user_quota": { 2019-12-09T09:27:20.392 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T09:27:20.392 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T09:27:20.392 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T09:27:20.392 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T09:27:20.392 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T09:27:20.393 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T09:27:20.393 INFO:teuthology.orchestra.run.smithi035.stdout: "temp_url_keys": [], 2019-12-09T09:27:20.393 INFO:teuthology.orchestra.run.smithi035.stdout: "type": "rgw", 2019-12-09T09:27:20.393 INFO:teuthology.orchestra.run.smithi035.stdout: "mfa_ids": [] 2019-12-09T09:27:20.393 INFO:teuthology.orchestra.run.smithi035.stdout:} 2019-12-09T09:27:20.393 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:27:20.426 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.355+0000 7f1c8d99bec0 20 remove_watcher() i=0 2019-12-09T09:27:20.426 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.355+0000 7f1c8d99bec0 2 removed watcher, disabling cache 2019-12-09T09:27:20.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.392+0000 7f1c8d99bec0 20 remove_watcher() i=1 2019-12-09T09:27:20.500 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.431+0000 7f1c8d99bec0 20 remove_watcher() i=2 2019-12-09T09:27:20.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.505+0000 7f1c8d99bec0 20 remove_watcher() i=3 2019-12-09T09:27:20.621 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.552+0000 7f1c8d99bec0 20 remove_watcher() i=4 2019-12-09T09:27:20.672 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.603+0000 7f1c8d99bec0 20 remove_watcher() i=5 2019-12-09T09:27:20.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.674+0000 7f1c8d99bec0 20 remove_watcher() i=6 2019-12-09T09:27:20.819 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:20.750+0000 7f1c8d99bec0 20 remove_watcher() i=7 2019-12-09T09:27:20.835 INFO:teuthology.orchestra.run.smithi035:> cat > /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2019-12-09T09:27:20.927 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2019-12-09T09:27:20.929 INFO:teuthology.orchestra.run.smithi035:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2019-12-09T09:27:21.018 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T09:27:21.147 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.078+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.177 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.108+0000 7f85a8922ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:21.177 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.108+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.185 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.116+0000 7f85a8922ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:21.185 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.116+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.190 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.121+0000 7f85a8922ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:21.190 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.121+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.198 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.129+0000 7f85a8922ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:21.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.361+0000 7f85a8922ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:27:21.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.361+0000 7f85a8922ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:27:21.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.361+0000 7f85a8922ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:27:21.432 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.361+0000 7f85a8922ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:27:21.432 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.361+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.375+0000 7f85a8922ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:21.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.375+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.396+0000 7f85a8922ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:27:21.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.396+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.483 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.414+0000 7f85a8922ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:21.483 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.414+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.500 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.431+0000 7f85a8922ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:27:21.500 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.431+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.506 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.437+0000 7f85a8922ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:21.506 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.437+0000 7f85a8922ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:27:21.506 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.437+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.518 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.449+0000 7f85a8922ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:21.518 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.449+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.531 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.462+0000 7f85a8922ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:21.532 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.462+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.552 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.483+0000 7f85a8922ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:27:21.553 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.483+0000 7f85a8922ec0 20 zonegroup default 2019-12-09T09:27:21.553 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.483+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.561 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.491+0000 7f85a8922ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:21.561 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.491+0000 7f85a8922ec0 10 Cannot find current period zone using local zone 2019-12-09T09:27:21.561 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.491+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.568 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.499+0000 7f85a8922ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:21.569 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.499+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.579 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.509+0000 7f85a8922ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:21.579 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.509+0000 7f85a8922ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:21.600 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.531+0000 7f85a8922ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:27:21.601 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.531+0000 7f85a8922ec0 20 zone default found 2019-12-09T09:27:21.601 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.531+0000 7f85a8922ec0 20 started sync module instance, tier type = 2019-12-09T09:27:21.601 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:21.531+0000 7f85a8922ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:27:22.070 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.000+0000 7f85a8922ec0 20 add_watcher() i=0 2019-12-09T09:27:22.070 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.000+0000 7f85a8922ec0 20 add_watcher() i=1 2019-12-09T09:27:22.070 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.000+0000 7f85a8922ec0 20 add_watcher() i=2 2019-12-09T09:27:22.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.000+0000 7f85a8922ec0 20 add_watcher() i=3 2019-12-09T09:27:22.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.000+0000 7f85a8922ec0 20 add_watcher() i=4 2019-12-09T09:27:22.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.000+0000 7f85a8922ec0 20 add_watcher() i=5 2019-12-09T09:27:22.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.000+0000 7f85a8922ec0 20 add_watcher() i=6 2019-12-09T09:27:22.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.046+0000 7f85a8922ec0 20 add_watcher() i=7 2019-12-09T09:27:22.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.046+0000 7f85a8922ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:27:22.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.046+0000 7f85667dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:27:22.121 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.052+0000 7f8595b4c700 0 -- 172.21.15.35:0/467869270 >> v1:172.21.15.22:6789/0 conn(0x55c158af98b0 legacy=0x55c158af6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T09:27:22.183 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:22.114+0000 7f859634d700 0 -- 172.21.15.35:0/1461654270 >> [v2:172.21.15.22:6800/8484,v1:172.21.15.22:6801/8484] conn(0x7f84d8022490 msgr2=0x7f84d8024980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:27:23.638 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.567+0000 7f84dcff9700 20 reqs_thread_entry: start 2019-12-09T09:27:23.638 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.567+0000 7f84ce7fc700 20 reqs_thread_entry: start 2019-12-09T09:27:23.638 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.568+0000 7f85a8922ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:27:23.639 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.568+0000 7f84ccff9700 20 reqs_thread_entry: start 2019-12-09T09:27:23.639 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.568+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:27:23.691 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.620+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:23.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.638+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:23.717 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.647+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:23.748 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.678+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:27:23.789 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.719+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:23.827 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.757+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:23.854 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.784+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:23.855 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.784+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='_YFD1SJ0Iy7__-5-cf2JlaBATgCfcwka', time=2019-12-09T11:26:50.146261+0000, chain.objs.size()=2 2019-12-09T09:27:23.855 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.784+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.tJpqgU4T0l5i_Uxh0S6-u_OH7u3Nugz_0 2019-12-09T09:27:23.855 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.784+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.tJpqgU4T0l5i_Uxh0S6-u_OH7u3Nugz_1 2019-12-09T09:27:23.930 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.859+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:24.030 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:23.959+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:27:24.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.045+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:24.154 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.084+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:24.170 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.100+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:24.227 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.157+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:27:24.302 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.233+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:24.361 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.292+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:24.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.352+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:24.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.352+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.71', time=2019-12-09T11:26:48.122761+0000, chain.objs.size()=4 2019-12-09T09:27:24.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.352+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5__multipart_foo.2~8JeeFpCRMEmnuJdes22iqQjf8D9FYSX.1 2019-12-09T09:27:24.423 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.352+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5__multipart_foo.2~8JeeFpCRMEmnuJdes22iqQjf8D9FYSX.2 2019-12-09T09:27:24.423 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.352+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5__multipart_foo.2~8JeeFpCRMEmnuJdes22iqQjf8D9FYSX.3 2019-12-09T09:27:24.423 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.352+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.5__multipart_foo.2~8JeeFpCRMEmnuJdes22iqQjf8D9FYSX.4 2019-12-09T09:27:24.572 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.502+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:24.709 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.640+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:27:24.760 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.691+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:24.799 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.729+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:24.833 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T09:27:24.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.114', time=2019-12-09T11:26:50.375288+0000, chain.objs.size()=2 2019-12-09T09:27:24.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.XvwKah-QkhNz47Rt2TrRD_c-Ng8gjL8_0 2019-12-09T09:27:24.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.XvwKah-QkhNz47Rt2TrRD_c-Ng8gjL8_1 2019-12-09T09:27:24.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='_yTkJfPzFsL4O6e-J2UN7YgOjcOEDNIS', time=2019-12-09T11:26:51.057658+0000, chain.objs.size()=1 2019-12-09T09:27:24.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.C3sueMTnd5jt2cyAjyZd5IQC_s0kRah_0 2019-12-09T09:27:24.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.224', time=2019-12-09T11:26:52.581841+0000, chain.objs.size()=5 2019-12-09T09:27:24.835 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-FROZEN.2~UMPNd6mrnAulpbRdvilLz8VyD4Ouqg0.1 2019-12-09T09:27:24.835 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_foo-FROZEN.2~UMPNd6mrnAulpbRdvilLz8VyD4Ouqg0.1_1 2019-12-09T09:27:24.835 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-FROZEN.2~UMPNd6mrnAulpbRdvilLz8VyD4Ouqg0.2 2019-12-09T09:27:24.835 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_foo-FROZEN.2~UMPNd6mrnAulpbRdvilLz8VyD4Ouqg0.2_1 2019-12-09T09:27:24.835 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.764+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-FROZEN.2~UMPNd6mrnAulpbRdvilLz8VyD4Ouqg0.3 2019-12-09T09:27:24.949 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:24.880+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:25.079 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.009+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:27:25.164 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.094+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:25.192 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.123+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:25.209 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.140+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:25.271 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.201+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:27:25.354 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.280+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:25.373 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.304+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:25.389 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.319+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:25.389 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.319+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='_O7CcwLFjZorZ1nKZDh7jOaxefi3T8Rd', time=2019-12-09T11:26:52.957258+0000, chain.objs.size()=3 2019-12-09T09:27:25.389 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.319+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.kH7j5uFmHNeaFAy31eUrF2toBfXqUax_0 2019-12-09T09:27:25.389 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.319+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.kH7j5uFmHNeaFAy31eUrF2toBfXqUax_1 2019-12-09T09:27:25.389 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.319+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.kH7j5uFmHNeaFAy31eUrF2toBfXqUax_2 2019-12-09T09:27:25.486 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.416+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:25.578 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.508+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:27:25.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.561+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:25.647 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.577+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:25.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.588+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:25.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.588+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='_Wr_T9xfJ8iCdEnymxS1_9iNk78pcoHp', time=2019-12-09T11:26:54.116881+0000, chain.objs.size()=2 2019-12-09T09:27:25.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.588+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.yexZEPJctKecUftQg2treo7SQylxLe5_1 2019-12-09T09:27:25.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.588+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.yexZEPJctKecUftQg2treo7SQylxLe5_2 2019-12-09T09:27:25.752 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.683+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:25.853 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.784+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:27:25.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.870+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:25.975 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.905+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:26.028 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.959+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:26.028 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.959+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.210', time=2019-12-09T11:26:56.860005+0000, chain.objs.size()=3 2019-12-09T09:27:26.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.959+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__multipart_foo-STANDARD.2~aXtuvljjNPO9cm-qWVyAShYHe3hRpHD.1 2019-12-09T09:27:26.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.959+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__multipart_foo-STANDARD.2~aXtuvljjNPO9cm-qWVyAShYHe3hRpHD.2 2019-12-09T09:27:26.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:25.959+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__multipart_foo-STANDARD.2~aXtuvljjNPO9cm-qWVyAShYHe3hRpHD.3 2019-12-09T09:27:26.210 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.140+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:26.320 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.250+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:27:26.387 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.318+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:26.406 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.337+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:26.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.350+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T09:27:26.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.350+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.20', time=2019-12-09T11:26:43.297922+0000, chain.objs.size()=4 2019-12-09T09:27:26.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.350+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2__multipart_foo.2~p_ruI4nTicD8sddTErv96BHRtk1HinD.1 2019-12-09T09:27:26.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.350+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2__multipart_foo.2~p_ruI4nTicD8sddTErv96BHRtk1HinD.2 2019-12-09T09:27:26.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.350+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2__multipart_foo.2~p_ruI4nTicD8sddTErv96BHRtk1HinD.3 2019-12-09T09:27:26.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.350+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.2__multipart_foo.2~p_ruI4nTicD8sddTErv96BHRtk1HinD.4 2019-12-09T09:27:26.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.350+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.112', time=2019-12-09T11:26:49.943179+0000, chain.objs.size()=2 2019-12-09T09:27:26.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.350+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.XvwKah-QkhNz47Rt2TrRD_c-Ng8gjL8_0 2019-12-09T09:27:26.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.350+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.XvwKah-QkhNz47Rt2TrRD_c-Ng8gjL8_1 2019-12-09T09:27:26.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.350+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='_A-d7hVRy2s1Q1kunFXYbfj-m3UCMzh-', time=2019-12-09T11:26:50.846594+0000, chain.objs.size()=1 2019-12-09T09:27:26.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.350+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.IxTe-0fQ0mui_yMmlTrhTR5jxTeH3vO_0 2019-12-09T09:27:26.600 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.530+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:26.709 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.639+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:27:26.762 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.692+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:26.790 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.721+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:26.799 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.730+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:26.842 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.772+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:27:26.900 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.830+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:26.923 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.853+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:26.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.873+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:27:26.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.873+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.40', time=2019-12-09T11:26:45.149996+0000, chain.objs.size()=4 2019-12-09T09:27:26.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.873+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3__multipart_foo.2~wTR8KUOu46d7XZRc-0_5h1LvLqDIP8Q.1 2019-12-09T09:27:26.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.873+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3__multipart_foo.2~wTR8KUOu46d7XZRc-0_5h1LvLqDIP8Q.2 2019-12-09T09:27:26.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.873+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3__multipart_foo.2~wTR8KUOu46d7XZRc-0_5h1LvLqDIP8Q.3 2019-12-09T09:27:26.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.873+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.3__multipart_foo.2~wTR8KUOu46d7XZRc-0_5h1LvLqDIP8Q.4 2019-12-09T09:27:26.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.873+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.242', time=2019-12-09T11:26:54.283782+0000, chain.objs.size()=3 2019-12-09T09:27:26.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.873+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-STANDARD.2~PCyO6A7SQX4-a-_yfJ-dEc_Rl6c4CoP.1 2019-12-09T09:27:26.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.873+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-STANDARD.2~PCyO6A7SQX4-a-_yfJ-dEc_Rl6c4CoP.2 2019-12-09T09:27:26.945 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:26.873+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-STANDARD.2~PCyO6A7SQX4-a-_yfJ-dEc_Rl6c4CoP.3 2019-12-09T09:27:27.156 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.087+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:27.256 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.187+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:27:27.327 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.257+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:27.353 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.283+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:27.367 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.298+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:27.415 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.345+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:27:27.463 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.394+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:27.480 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.411+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:27.505 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.436+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:27.545 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.476+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:27:27.598 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.528+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:27.614 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.544+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:27.647 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.577+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T09:27:27.647 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.577+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.109', time=2019-12-09T11:26:49.335476+0000, chain.objs.size()=2 2019-12-09T09:27:27.647 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.577+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.kb8_QBOooSUlHYqSoD_5XX7F_Q4URpY_0 2019-12-09T09:27:27.648 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.577+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.kb8_QBOooSUlHYqSoD_5XX7F_Q4URpY_1 2019-12-09T09:27:27.648 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.577+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='_MeqjdK6ZAkz9cutu_lg2TrrpHMIq3nX', time=2019-12-09T11:26:53.235937+0000, chain.objs.size()=2 2019-12-09T09:27:27.648 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.577+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.OG9kxxf9cdsqnz3twZnO4mtNlQk10IK_1 2019-12-09T09:27:27.648 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.577+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.OG9kxxf9cdsqnz3twZnO4mtNlQk10IK_2 2019-12-09T09:27:27.648 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.577+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='_bSB6s2pvDxBc_Hn9b2Svio5PoxtdiKt', time=2019-12-09T11:26:54.508420+0000, chain.objs.size()=1 2019-12-09T09:27:27.648 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.577+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.7r-DTr9-aZdNSgSYVaV9w8UaattUpJr_0 2019-12-09T09:27:27.762 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.693+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:27.875 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.805+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:27:27.937 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.867+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:27.963 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.893+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:27.979 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.909+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:27.979 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.909+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.84', time=2019-12-09T11:26:57.675389+0000, chain.objs.size()=3 2019-12-09T09:27:27.979 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.909+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6__shadow_.Mut878wDah6VJA99FKVSQ4UJy8ZRAyH_0 2019-12-09T09:27:27.980 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.909+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6__shadow_.Mut878wDah6VJA99FKVSQ4UJy8ZRAyH_1 2019-12-09T09:27:27.980 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:27.909+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6__shadow_.Mut878wDah6VJA99FKVSQ4UJy8ZRAyH_2 2019-12-09T09:27:28.082 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.012+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:28.164 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.094+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:27:28.211 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.138+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:28.225 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.156+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:28.234 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.165+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:28.268 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.199+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:27:28.312 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.242+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:28.333 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.260+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:28.340 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.270+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:28.380 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.310+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:27:28.441 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.372+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:28.471 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.401+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:28.493 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.423+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:28.493 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.423+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.203', time=2019-12-09T11:26:56.698287+0000, chain.objs.size()=5 2019-12-09T09:27:28.493 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.423+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__multipart_foo-LUKEWARM.2~upTfruSQMn15FcKy9rN5r9kF_joi_jK.1 2019-12-09T09:27:28.493 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.423+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__shadow_foo-LUKEWARM.2~upTfruSQMn15FcKy9rN5r9kF_joi_jK.1_1 2019-12-09T09:27:28.494 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.423+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__multipart_foo-LUKEWARM.2~upTfruSQMn15FcKy9rN5r9kF_joi_jK.2 2019-12-09T09:27:28.494 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.423+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__shadow_foo-LUKEWARM.2~upTfruSQMn15FcKy9rN5r9kF_joi_jK.2_1 2019-12-09T09:27:28.494 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.423+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__multipart_foo-LUKEWARM.2~upTfruSQMn15FcKy9rN5r9kF_joi_jK.3 2019-12-09T09:27:28.583 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.513+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:28.679 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.609+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:27:28.737 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.667+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:28.756 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.686+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:28.768 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.699+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:28.806 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.736+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:27:28.856 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.786+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:28.873 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.804+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:28.892 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T09:27:28.892 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='_PubZ4ROdivGxPas0dy6OdH-WtOrxX6y', time=2019-12-09T11:26:49.626848+0000, chain.objs.size()=2 2019-12-09T09:27:28.892 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.SvgNLuAyw037K8k1AI9ayGOv4J8zubz_0 2019-12-09T09:27:28.893 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.SvgNLuAyw037K8k1AI9ayGOv4J8zubz_1 2019-12-09T09:27:28.893 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.225', time=2019-12-09T11:26:52.840195+0000, chain.objs.size()=5 2019-12-09T09:27:28.893 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-FROZEN.2~UMPNd6mrnAulpbRdvilLz8VyD4Ouqg0.1 2019-12-09T09:27:28.893 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_foo-FROZEN.2~UMPNd6mrnAulpbRdvilLz8VyD4Ouqg0.1_1 2019-12-09T09:27:28.893 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-FROZEN.2~UMPNd6mrnAulpbRdvilLz8VyD4Ouqg0.2 2019-12-09T09:27:28.893 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_foo-FROZEN.2~UMPNd6mrnAulpbRdvilLz8VyD4Ouqg0.2_1 2019-12-09T09:27:28.894 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-FROZEN.2~UMPNd6mrnAulpbRdvilLz8VyD4Ouqg0.3 2019-12-09T09:27:28.894 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='_Xly0KU9dhfiB6N97XyLOKe-FZJwggVT', time=2019-12-09T11:26:53.809126+0000, chain.objs.size()=3 2019-12-09T09:27:28.894 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.ZnwB0jQJUjjLIZI-9Bz58oEawJSVy90_0 2019-12-09T09:27:28.894 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.ZnwB0jQJUjjLIZI-9Bz58oEawJSVy90_1 2019-12-09T09:27:28.894 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.822+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.ZnwB0jQJUjjLIZI-9Bz58oEawJSVy90_2 2019-12-09T09:27:29.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:28.960+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:29.105 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.036+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:27:29.180 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.110+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:29.200 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.130+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:29.223 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.152+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:27:29.223 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.152+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.233', time=2019-12-09T11:26:53.450551+0000, chain.objs.size()=5 2019-12-09T09:27:29.223 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.152+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-LUKEWARM.2~mk8hH0mSreOeOMxfka5VcGyFos7kBJ0.1 2019-12-09T09:27:29.223 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.153+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_foo-LUKEWARM.2~mk8hH0mSreOeOMxfka5VcGyFos7kBJ0.1_1 2019-12-09T09:27:29.223 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.153+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-LUKEWARM.2~mk8hH0mSreOeOMxfka5VcGyFos7kBJ0.2 2019-12-09T09:27:29.223 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.153+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_foo-LUKEWARM.2~mk8hH0mSreOeOMxfka5VcGyFos7kBJ0.2_1 2019-12-09T09:27:29.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.153+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-LUKEWARM.2~mk8hH0mSreOeOMxfka5VcGyFos7kBJ0.3 2019-12-09T09:27:29.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.153+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.235', time=2019-12-09T11:26:53.891350+0000, chain.objs.size()=5 2019-12-09T09:27:29.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.153+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-LUKEWARM.2~mk8hH0mSreOeOMxfka5VcGyFos7kBJ0.1 2019-12-09T09:27:29.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.153+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_foo-LUKEWARM.2~mk8hH0mSreOeOMxfka5VcGyFos7kBJ0.1_1 2019-12-09T09:27:29.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.153+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-LUKEWARM.2~mk8hH0mSreOeOMxfka5VcGyFos7kBJ0.2 2019-12-09T09:27:29.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.153+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_foo-LUKEWARM.2~mk8hH0mSreOeOMxfka5VcGyFos7kBJ0.2_1 2019-12-09T09:27:29.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.153+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-LUKEWARM.2~mk8hH0mSreOeOMxfka5VcGyFos7kBJ0.3 2019-12-09T09:27:29.425 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.355+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:29.502 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.432+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:27:29.540 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.470+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:29.556 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.486+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:29.572 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.502+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:27:29.572 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.502+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.245', time=2019-12-09T11:26:54.884147+0000, chain.objs.size()=3 2019-12-09T09:27:29.573 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.502+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-STANDARD.2~PCyO6A7SQX4-a-_yfJ-dEc_Rl6c4CoP.1 2019-12-09T09:27:29.573 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.502+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-STANDARD.2~PCyO6A7SQX4-a-_yfJ-dEc_Rl6c4CoP.2 2019-12-09T09:27:29.573 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.502+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__multipart_foo-STANDARD.2~PCyO6A7SQX4-a-_yfJ-dEc_Rl6c4CoP.3 2019-12-09T09:27:29.573 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.502+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.196', time=2019-12-09T11:26:56.420913+0000, chain.objs.size()=5 2019-12-09T09:27:29.573 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.502+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__multipart_foo-FROZEN.2~Dcc1ilAaZk0JQebJLajNK7IwYRc4G-L.1 2019-12-09T09:27:29.573 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.502+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__shadow_foo-FROZEN.2~Dcc1ilAaZk0JQebJLajNK7IwYRc4G-L.1_1 2019-12-09T09:27:29.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.502+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__multipart_foo-FROZEN.2~Dcc1ilAaZk0JQebJLajNK7IwYRc4G-L.2 2019-12-09T09:27:29.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.502+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__shadow_foo-FROZEN.2~Dcc1ilAaZk0JQebJLajNK7IwYRc4G-L.2_1 2019-12-09T09:27:29.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.502+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.8__multipart_foo-FROZEN.2~Dcc1ilAaZk0JQebJLajNK7IwYRc4G-L.3 2019-12-09T09:27:29.754 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.684+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.772+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:27:29.893 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.823+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:29.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.839+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:29.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.871+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:29.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.871+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.55', time=2019-12-09T11:26:46.692259+0000, chain.objs.size()=4 2019-12-09T09:27:29.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.871+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4__multipart_foo.2~n2Vpiu4gFEdec1S18kJ6y6ksx95Ojia.1 2019-12-09T09:27:29.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.871+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4__multipart_foo.2~n2Vpiu4gFEdec1S18kJ6y6ksx95Ojia.2 2019-12-09T09:27:29.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.871+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4__multipart_foo.2~n2Vpiu4gFEdec1S18kJ6y6ksx95Ojia.3 2019-12-09T09:27:29.942 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:29.871+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.4__multipart_foo.2~n2Vpiu4gFEdec1S18kJ6y6ksx95Ojia.4 2019-12-09T09:27:30.073 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.003+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:30.161 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.091+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:27:30.211 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.141+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:30.232 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.162+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:30.240 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.170+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:30.305 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.235+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:27:30.366 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.296+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:30.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.315+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:30.393 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.323+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:30.437 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.367+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:27:30.492 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.422+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:30.518 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.443+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:30.524 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.454+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:30.524 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.454+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='__KSkJPOzsJmV2gHP12D92Wv4IR8vdTy', time=2019-12-09T11:26:54.686323+0000, chain.objs.size()=1 2019-12-09T09:27:30.524 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.454+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.9__shadow_.wGhfwyv_ugE1Ss76_R--B_630VqUCYx_0 2019-12-09T09:27:30.576 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.506+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:30.661 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.591+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:27:30.705 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.635+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:30.726 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.656+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:30.733 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.663+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:30.784 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.714+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:27:30.848 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.778+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:30.872 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.802+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:30.897 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.828+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:30.898 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.828+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='_HW0vY1jL3QytnL3rSqlKMo1bWIHZ_c_', time=2019-12-09T11:26:49.745521+0000, chain.objs.size()=1 2019-12-09T09:27:30.898 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.828+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.c9xOG1gorXbsx0on9hV_LW4jAgkBgVV_1 2019-12-09T09:27:31.005 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:30.935+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:31.098 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.028+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:27:31.170 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.100+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:31.191 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.120+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:31.205 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.135+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:31.205 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.135+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='_0uP-qtn5iz4TxndlB9hXNZah1tBQh6P', time=2019-12-09T11:26:50.610939+0000, chain.objs.size()=1 2019-12-09T09:27:31.205 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.135+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.-QvxOB9s_OCih10A28AzT6gY3JQoS8N_1 2019-12-09T09:27:31.273 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.203+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:31.418 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.349+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:27:31.487 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.417+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:31.513 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.443+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:31.529 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.459+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:31.530 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.459+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.108', time=2019-12-09T11:26:49.140775+0000, chain.objs.size()=2 2019-12-09T09:27:31.530 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.459+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.kb8_QBOooSUlHYqSoD_5XX7F_Q4URpY_0 2019-12-09T09:27:31.530 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.459+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.7__shadow_.kb8_QBOooSUlHYqSoD_5XX7F_Q4URpY_1 2019-12-09T09:27:31.596 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.526+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:31.701 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.631+0000 7f85a8922ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:27:31.792 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.723+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:31.831 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.761+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:31.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.781+0000 7f85a8922ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:27:31.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.781+0000 7f85a8922ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.85', time=2019-12-09T11:26:57.930323+0000, chain.objs.size()=3 2019-12-09T09:27:31.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.781+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6__shadow_.pUeTtn7Dwkd6jLIy6AgtqaKlkfZlSH5_0 2019-12-09T09:27:31.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.781+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6__shadow_.pUeTtn7Dwkd6jLIy6AgtqaKlkfZlSH5_1 2019-12-09T09:27:31.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.781+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.6__shadow_.pUeTtn7Dwkd6jLIy6AgtqaKlkfZlSH5_2 2019-12-09T09:27:31.981 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:31.911+0000 7f85a8922ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:27:32.165 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.095+0000 7f85a8922ec0 20 remove_watcher() i=0 2019-12-09T09:27:32.166 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.095+0000 7f85a8922ec0 2 removed watcher, disabling cache 2019-12-09T09:27:32.234 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.164+0000 7f85a8922ec0 20 remove_watcher() i=1 2019-12-09T09:27:32.282 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.212+0000 7f85a8922ec0 20 remove_watcher() i=2 2019-12-09T09:27:32.355 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.285+0000 7f85a8922ec0 20 remove_watcher() i=3 2019-12-09T09:27:32.405 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.335+0000 7f85a8922ec0 20 remove_watcher() i=4 2019-12-09T09:27:32.463 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.393+0000 7f85a8922ec0 20 remove_watcher() i=5 2019-12-09T09:27:32.501 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.431+0000 7f85a8922ec0 20 remove_watcher() i=6 2019-12-09T09:27:32.572 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.502+0000 7f85a8922ec0 20 remove_watcher() i=7 2019-12-09T09:27:32.583 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T09:27:32.739 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.669+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:32.762 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.692+0000 7f0c3f061ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:32.762 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.692+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:32.771 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.701+0000 7f0c3f061ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:32.771 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.701+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:32.777 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.707+0000 7f0c3f061ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:32.777 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.707+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:32.784 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.715+0000 7f0c3f061ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:33.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.976+0000 7f0c3f061ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:27:33.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.976+0000 7f0c3f061ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:27:33.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.976+0000 7f0c3f061ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:27:33.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.976+0000 7f0c3f061ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:27:33.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.976+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.062 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.992+0000 7f0c3f061ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:33.062 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:32.992+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.080 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.010+0000 7f0c3f061ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:27:33.081 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.010+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.094 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.024+0000 7f0c3f061ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:33.094 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.024+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.045+0000 7f0c3f061ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:27:33.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.045+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.159 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.089+0000 7f0c3f061ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:33.159 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.089+0000 7f0c3f061ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:27:33.159 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.089+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.173 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.103+0000 7f0c3f061ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:33.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.103+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.188 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.118+0000 7f0c3f061ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:33.188 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.118+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.211 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.140+0000 7f0c3f061ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:27:33.211 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.140+0000 7f0c3f061ec0 20 zonegroup default 2019-12-09T09:27:33.212 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.140+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.153+0000 7f0c3f061ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:33.225 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.153+0000 7f0c3f061ec0 10 Cannot find current period zone using local zone 2019-12-09T09:27:33.225 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.153+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.250 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.179+0000 7f0c3f061ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:27:33.250 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.179+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.266 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.195+0000 7f0c3f061ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:27:33.266 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.195+0000 7f0c3f061ec0 20 rados->read ofs=0 len=0 2019-12-09T09:27:33.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.213+0000 7f0c3f061ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:27:33.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.213+0000 7f0c3f061ec0 20 zone default found 2019-12-09T09:27:33.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.213+0000 7f0c3f061ec0 20 started sync module instance, tier type = 2019-12-09T09:27:33.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.213+0000 7f0c3f061ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:27:33.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.551+0000 7f0c3f061ec0 20 add_watcher() i=0 2019-12-09T09:27:33.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.551+0000 7f0c3f061ec0 20 add_watcher() i=1 2019-12-09T09:27:33.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.551+0000 7f0c3f061ec0 20 add_watcher() i=2 2019-12-09T09:27:33.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.551+0000 7f0c3f061ec0 20 add_watcher() i=3 2019-12-09T09:27:33.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.551+0000 7f0c3f061ec0 20 add_watcher() i=4 2019-12-09T09:27:33.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.551+0000 7f0c3f061ec0 20 add_watcher() i=5 2019-12-09T09:27:33.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.551+0000 7f0c3f061ec0 20 add_watcher() i=6 2019-12-09T09:27:33.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.581+0000 7f0c3f061ec0 20 add_watcher() i=7 2019-12-09T09:27:33.653 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.581+0000 7f0c3f061ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:27:33.653 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.582+0000 7f0bfcfe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:27:33.657 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.586+0000 7f0c2ca8c700 0 -- 172.21.15.35:0/4230296233 >> v1:172.21.15.35:6789/0 conn(0x56323cef2260 legacy=0x56323cf00710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T09:27:33.701 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:33.630+0000 7f0c2ba8a700 0 -- 172.21.15.35:0/3971952225 >> [v2:172.21.15.35:6808/9789,v1:172.21.15.35:6809/9789] conn(0x56323cf14900 msgr2=0x56323cf16d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:27:35.458 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.388+0000 7f0b6effd700 20 reqs_thread_entry: start 2019-12-09T09:27:35.458 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.388+0000 7f0b6cff9700 20 reqs_thread_entry: start 2019-12-09T09:27:35.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.388+0000 7f0c3f061ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:27:35.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.388+0000 7f0b62ffd700 20 reqs_thread_entry: start 2019-12-09T09:27:35.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.388+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:27:35.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.439+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:35.527 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.457+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:35.536 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.466+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:35.621 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.551+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:27:35.663 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.593+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:35.680 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.610+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:35.689 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.619+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:35.836 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.766+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:27:35.893 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.823+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:35.912 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.842+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:35.921 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.851+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:35.954 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.883+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:27:36.049 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:35.979+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.084 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.013+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:36.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.027+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.135 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.064+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:27:36.179 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.109+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.200 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.130+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:36.212 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.142+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.242 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.172+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:27:36.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.215+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.304 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.234+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:36.313 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.243+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.345 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.275+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:27:36.418 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.344+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.445 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.375+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:36.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.394+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.523 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.453+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:27:36.576 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.502+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.593 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.522+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:36.603 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.533+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.634 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.564+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:27:36.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.623+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.717 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.646+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:36.729 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.659+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.775 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.705+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:27:36.822 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.752+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.838 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.768+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:36.847 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.777+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.885 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.814+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:27:36.933 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.863+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:36.951 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.881+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:36.962 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.892+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.932+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:27:37.055 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:36.984+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.074 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.004+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:37.083 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.013+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.047+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:27:37.180 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.109+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.195 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.125+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:37.204 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.134+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.232 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.162+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:27:37.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.211+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.237+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:37.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.249+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.364 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.294+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:27:37.448 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.377+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.476 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.405+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:37.494 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.424+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.538 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.468+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:27:37.588 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.517+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.538+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:37.621 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.550+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.585+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:27:37.745 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.675+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.786 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.716+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:37.812 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.742+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.871 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.800+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:27:37.929 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.849+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:37.947 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.876+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:37.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.886+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.009 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.938+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:27:38.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:37.986+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.005+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:38.084 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.013+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.127 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.056+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:27:38.202 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.131+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.237 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.166+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:38.255 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.184+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.230+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:27:38.372 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.301+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.400 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.330+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:38.413 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.342+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.453 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.383+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:27:38.526 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.456+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.568 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.498+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:38.586 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.516+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.641 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.570+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:27:38.706 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.635+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.741 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.669+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:38.753 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.683+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.730+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:27:38.863 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.792+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.880 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.810+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:38.893 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.823+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:38.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.856+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:27:38.992 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.922+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.941+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:39.022 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:38.950+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.001+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:27:39.149 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.078+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.178 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.107+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:39.195 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.124+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.244 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.173+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:27:39.317 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.247+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.279+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:39.360 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.289+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.410 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.340+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:27:39.515 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.445+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.545 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.475+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:39.561 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.490+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.603 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.532+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:27:39.647 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.577+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.664 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.593+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:39.677 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.606+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.712 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.641+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:27:39.794 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.723+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.838 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.768+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:39.863 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.793+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:39.918 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.847+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:27:40.008 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.937+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:40.037 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.966+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:40.049 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:39.978+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:40.084 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.014+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:27:40.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.075+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:40.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.103+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:27:40.194 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.123+0000 7f0c3f061ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:27:40.306 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.236+0000 7f0c3f061ec0 20 remove_watcher() i=0 2019-12-09T09:27:40.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.236+0000 7f0c3f061ec0 2 removed watcher, disabling cache 2019-12-09T09:27:40.347 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.276+0000 7f0c3f061ec0 20 remove_watcher() i=1 2019-12-09T09:27:40.387 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.316+0000 7f0c3f061ec0 20 remove_watcher() i=2 2019-12-09T09:27:40.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.385+0000 7f0c3f061ec0 20 remove_watcher() i=3 2019-12-09T09:27:40.507 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.437+0000 7f0c3f061ec0 20 remove_watcher() i=4 2019-12-09T09:27:40.567 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.496+0000 7f0c3f061ec0 20 remove_watcher() i=5 2019-12-09T09:27:40.624 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.553+0000 7f0c3f061ec0 20 remove_watcher() i=6 2019-12-09T09:27:40.682 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:27:40.611+0000 7f0c3f061ec0 20 remove_watcher() i=7 2019-12-09T09:27:40.693 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2019-12-09T09:27:41.326 INFO:teuthology.orchestra.run.smithi035.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2019-12-09T09:27:44.926 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.pom 2019-12-09T09:27:45.126 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.jar 2019-12-09T09:27:46.228 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :clean UP-TO-DATE 2019-12-09T09:27:46.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-bom/1.11.549/aws-java-sdk-bom-1.11.549.pom 2019-12-09T09:27:46.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.549/aws-java-sdk-pom-1.11.549.pom 2019-12-09T09:27:46.527 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.pom 2019-12-09T09:27:46.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.pom 2019-12-09T09:27:46.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.pom 2019-12-09T09:27:46.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.pom 2019-12-09T09:27:46.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.pom 2019-12-09T09:27:46.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.pom 2019-12-09T09:27:46.627 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.pom 2019-12-09T09:27:46.627 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/3/oss-parent-3.pom 2019-12-09T09:27:46.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-parent/2.44.0/selenium-parent-2.44.0.pom 2019-12-09T09:27:46.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.pom 2019-12-09T09:27:46.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.pom 2019-12-09T09:27:46.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.pom 2019-12-09T09:27:46.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.pom 2019-12-09T09:27:46.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.pom 2019-12-09T09:27:46.630 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.pom 2019-12-09T09:27:46.630 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.5/httpcomponents-client-4.5.5.pom 2019-12-09T09:27:46.630 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.2/jackson-parent-2.6.2.pom 2019-12-09T09:27:46.727 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/28/commons-parent-28.pom 2019-12-09T09:27:46.727 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/24/oss-parent-24.pom 2019-12-09T09:27:46.727 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/10/httpcomponents-parent-10.pom 2019-12-09T09:27:46.728 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2019-12-09T09:27:46.728 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2019-12-09T09:27:46.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.pom 2019-12-09T09:27:46.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.pom 2019-12-09T09:27:46.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.pom 2019-12-09T09:27:46.830 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.pom 2019-12-09T09:27:46.830 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.pom 2019-12-09T09:27:46.830 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.pom 2019-12-09T09:27:46.830 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.pom 2019-12-09T09:27:46.830 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.pom 2019-12-09T09:27:46.830 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.pom 2019-12-09T09:27:46.830 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.pom 2019-12-09T09:27:46.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.pom 2019-12-09T09:27:46.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.pom 2019-12-09T09:27:46.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.pom 2019-12-09T09:27:46.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.pom 2019-12-09T09:27:46.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.pom 2019-12-09T09:27:46.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.pom 2019-12-09T09:27:46.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.pom 2019-12-09T09:27:46.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.pom 2019-12-09T09:27:46.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.pom 2019-12-09T09:27:46.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.pom 2019-12-09T09:27:46.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.pom 2019-12-09T09:27:46.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.pom 2019-12-09T09:27:46.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.pom 2019-12-09T09:27:46.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.pom 2019-12-09T09:27:46.833 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.pom 2019-12-09T09:27:46.833 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.pom 2019-12-09T09:27:46.833 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.pom 2019-12-09T09:27:46.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.pom 2019-12-09T09:27:46.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.pom 2019-12-09T09:27:46.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.pom 2019-12-09T09:27:46.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.pom 2019-12-09T09:27:46.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.pom 2019-12-09T09:27:46.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.pom 2019-12-09T09:27:46.931 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.pom 2019-12-09T09:27:46.931 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.pom 2019-12-09T09:27:46.932 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.pom 2019-12-09T09:27:46.932 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.pom 2019-12-09T09:27:46.932 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.pom 2019-12-09T09:27:46.932 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.pom 2019-12-09T09:27:46.932 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.pom 2019-12-09T09:27:46.932 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.pom 2019-12-09T09:27:46.932 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.pom 2019-12-09T09:27:46.933 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.pom 2019-12-09T09:27:46.933 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.pom 2019-12-09T09:27:46.933 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.pom 2019-12-09T09:27:46.933 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.pom 2019-12-09T09:27:46.933 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.pom 2019-12-09T09:27:46.933 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.pom 2019-12-09T09:27:46.933 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.pom 2019-12-09T09:27:46.934 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.pom 2019-12-09T09:27:46.934 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.pom 2019-12-09T09:27:46.934 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.pom 2019-12-09T09:27:46.935 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.pom 2019-12-09T09:27:46.935 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.pom 2019-12-09T09:27:46.935 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.pom 2019-12-09T09:27:46.936 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.pom 2019-12-09T09:27:46.936 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.pom 2019-12-09T09:27:47.028 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.pom 2019-12-09T09:27:47.028 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.pom 2019-12-09T09:27:47.028 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.pom 2019-12-09T09:27:47.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.pom 2019-12-09T09:27:47.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.pom 2019-12-09T09:27:47.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.pom 2019-12-09T09:27:47.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.pom 2019-12-09T09:27:47.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.pom 2019-12-09T09:27:47.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.pom 2019-12-09T09:27:47.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.pom 2019-12-09T09:27:47.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.pom 2019-12-09T09:27:47.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.pom 2019-12-09T09:27:47.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.pom 2019-12-09T09:27:47.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.pom 2019-12-09T09:27:47.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.pom 2019-12-09T09:27:47.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.pom 2019-12-09T09:27:47.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.pom 2019-12-09T09:27:47.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.pom 2019-12-09T09:27:47.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.pom 2019-12-09T09:27:47.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.pom 2019-12-09T09:27:47.032 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.pom 2019-12-09T09:27:47.032 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.pom 2019-12-09T09:27:47.032 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.pom 2019-12-09T09:27:47.033 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.pom 2019-12-09T09:27:47.033 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.pom 2019-12-09T09:27:47.033 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.pom 2019-12-09T09:27:47.034 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.pom 2019-12-09T09:27:47.034 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.pom 2019-12-09T09:27:47.034 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.pom 2019-12-09T09:27:47.034 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.pom 2019-12-09T09:27:47.034 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.pom 2019-12-09T09:27:47.034 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.pom 2019-12-09T09:27:47.035 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.pom 2019-12-09T09:27:47.035 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.pom 2019-12-09T09:27:47.035 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.pom 2019-12-09T09:27:47.035 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.pom 2019-12-09T09:27:47.035 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.pom 2019-12-09T09:27:47.129 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.pom 2019-12-09T09:27:47.130 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.pom 2019-12-09T09:27:47.130 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.pom 2019-12-09T09:27:47.130 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.pom 2019-12-09T09:27:47.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.pom 2019-12-09T09:27:47.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.pom 2019-12-09T09:27:47.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.pom 2019-12-09T09:27:47.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.pom 2019-12-09T09:27:47.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.pom 2019-12-09T09:27:47.132 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.pom 2019-12-09T09:27:47.132 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.pom 2019-12-09T09:27:47.132 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.pom 2019-12-09T09:27:47.132 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.pom 2019-12-09T09:27:47.133 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.pom 2019-12-09T09:27:47.133 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.pom 2019-12-09T09:27:47.133 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.pom 2019-12-09T09:27:47.133 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.pom 2019-12-09T09:27:47.134 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.pom 2019-12-09T09:27:47.134 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.pom 2019-12-09T09:27:47.134 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.pom 2019-12-09T09:27:47.134 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.pom 2019-12-09T09:27:47.135 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.pom 2019-12-09T09:27:47.135 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.pom 2019-12-09T09:27:47.135 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.pom 2019-12-09T09:27:47.135 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.pom 2019-12-09T09:27:47.136 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.pom 2019-12-09T09:27:47.136 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.pom 2019-12-09T09:27:47.136 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.pom 2019-12-09T09:27:47.136 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.pom 2019-12-09T09:27:47.137 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.pom 2019-12-09T09:27:47.137 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.pom 2019-12-09T09:27:47.137 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.pom 2019-12-09T09:27:47.137 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.pom 2019-12-09T09:27:47.137 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.pom 2019-12-09T09:27:47.138 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.pom 2019-12-09T09:27:47.138 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.pom 2019-12-09T09:27:47.138 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.pom 2019-12-09T09:27:47.138 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.pom 2019-12-09T09:27:47.139 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.pom 2019-12-09T09:27:47.139 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.pom 2019-12-09T09:27:47.229 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.pom 2019-12-09T09:27:47.229 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.pom 2019-12-09T09:27:47.229 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.pom 2019-12-09T09:27:47.229 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.pom 2019-12-09T09:27:47.229 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.pom 2019-12-09T09:27:47.229 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.pom 2019-12-09T09:27:47.230 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.pom 2019-12-09T09:27:47.230 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.pom 2019-12-09T09:27:47.230 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.pom 2019-12-09T09:27:47.230 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.pom 2019-12-09T09:27:47.230 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.pom 2019-12-09T09:27:47.230 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.pom 2019-12-09T09:27:47.230 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.pom 2019-12-09T09:27:47.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.pom 2019-12-09T09:27:47.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.pom 2019-12-09T09:27:47.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.pom 2019-12-09T09:27:47.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.pom 2019-12-09T09:27:47.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.pom 2019-12-09T09:27:47.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.pom 2019-12-09T09:27:47.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.pom 2019-12-09T09:27:47.232 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.pom 2019-12-09T09:27:47.232 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.pom 2019-12-09T09:27:47.232 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.pom 2019-12-09T09:27:47.232 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.pom 2019-12-09T09:27:47.232 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.pom 2019-12-09T09:27:47.232 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.pom 2019-12-09T09:27:47.232 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.pom 2019-12-09T09:27:47.233 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.pom 2019-12-09T09:27:47.233 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.pom 2019-12-09T09:27:47.233 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.pom 2019-12-09T09:27:47.233 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.pom 2019-12-09T09:27:47.233 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.22/aws-java-sdk-pom-1.11.22.pom 2019-12-09T09:27:47.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.pom 2019-12-09T09:27:47.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.pom 2019-12-09T09:27:47.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.pom 2019-12-09T09:27:47.330 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.pom 2019-12-09T09:27:47.330 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.pom 2019-12-09T09:27:47.330 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.pom 2019-12-09T09:27:47.330 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.pom 2019-12-09T09:27:47.331 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.pom 2019-12-09T09:27:47.331 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.pom 2019-12-09T09:27:47.331 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/project/6.1.9/project-6.1.9.pom 2019-12-09T09:27:47.331 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.pom 2019-12-09T09:27:47.331 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.pom 2019-12-09T09:27:47.331 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/32/commons-parent-32.pom 2019-12-09T09:27:47.332 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.pom 2019-12-09T09:27:47.332 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.pom 2019-12-09T09:27:47.332 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.pom 2019-12-09T09:27:47.433 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/beanshell/2.0b4/beanshell-2.0b4.pom 2019-12-09T09:27:47.433 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.pom 2019-12-09T09:27:47.435 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/34/commons-parent-34.pom 2019-12-09T09:27:47.435 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.pom 2019-12-09T09:27:47.435 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/35/commons-parent-35.pom 2019-12-09T09:27:47.435 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2019-12-09T09:27:47.436 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.pom 2019-12-09T09:27:47.436 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.9/httpcomponents-core-4.4.9.pom 2019-12-09T09:27:47.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/9/httpcomponents-parent-9.pom 2019-12-09T09:27:47.529 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.pom 2019-12-09T09:27:47.529 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.pom 2019-12-09T09:27:47.529 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.1/jackson-parent-2.6.1.pom 2019-12-09T09:27:47.529 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/23/oss-parent-23.pom 2019-12-09T09:27:47.530 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.pom 2019-12-09T09:27:47.530 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.pom 2019-12-09T09:27:47.530 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-parent/4.1.17.Final/netty-parent-4.1.17.Final.pom 2019-12-09T09:27:47.627 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2019-12-09T09:27:47.627 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.pom 2019-12-09T09:27:47.627 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.pom 2019-12-09T09:27:47.627 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.pom 2019-12-09T09:27:47.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.pom 2019-12-09T09:27:47.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.pom 2019-12-09T09:27:47.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.pom 2019-12-09T09:27:47.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.pom 2019-12-09T09:27:47.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/6/oss-parent-6.pom 2019-12-09T09:27:47.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.pom 2019-12-09T09:27:47.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.pom 2019-12-09T09:27:47.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.pom 2019-12-09T09:27:47.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.pom 2019-12-09T09:27:47.727 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.pom 2019-12-09T09:27:47.727 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.pom 2019-12-09T09:27:47.727 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.pom 2019-12-09T09:27:47.727 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/25/commons-parent-25.pom 2019-12-09T09:27:47.729 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/17/commons-parent-17.pom 2019-12-09T09:27:47.729 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/7/apache-7.pom 2019-12-09T09:27:47.729 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/9/apache-9.pom 2019-12-09T09:27:47.729 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.pom 2019-12-09T09:27:47.729 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.pom 2019-12-09T09:27:47.730 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.pom 2019-12-09T09:27:47.827 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2019-12-09T09:27:47.827 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.pom 2019-12-09T09:27:47.827 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.pom 2019-12-09T09:27:47.830 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.pom 2019-12-09T09:27:47.830 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.pom 2019-12-09T09:27:47.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.pom 2019-12-09T09:27:47.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.pom 2019-12-09T09:27:47.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/18.0/guava-parent-18.0.pom 2019-12-09T09:27:47.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.pom 2019-12-09T09:27:47.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.pom 2019-12-09T09:27:47.831 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.pom 2019-12-09T09:27:47.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.pom 2019-12-09T09:27:47.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.pom 2019-12-09T09:27:47.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.pom 2019-12-09T09:27:47.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.pom 2019-12-09T09:27:47.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.pom 2019-12-09T09:27:47.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/4/apache-4.pom 2019-12-09T09:27:47.832 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/9/commons-parent-9.pom 2019-12-09T09:27:47.833 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/33/commons-parent-33.pom 2019-12-09T09:27:47.834 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.pom 2019-12-09T09:27:47.834 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.3.3/httpcomponents-client-4.3.3.pom 2019-12-09T09:27:47.927 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-project/8.1.15.v20140411/jetty-project-8.1.15.v20140411.pom 2019-12-09T09:27:47.927 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/project/7/project-7.pom 2019-12-09T09:27:47.927 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-parent/20/jetty-parent-20.pom 2019-12-09T09:27:47.927 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.pom 2019-12-09T09:27:47.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.pom 2019-12-09T09:27:47.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.pom 2019-12-09T09:27:47.929 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.pom 2019-12-09T09:27:47.929 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.pom 2019-12-09T09:27:47.929 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.pom 2019-12-09T09:27:48.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.jar 2019-12-09T09:27:48.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.jar 2019-12-09T09:27:48.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.jar 2019-12-09T09:27:48.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.jar 2019-12-09T09:27:48.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.jar 2019-12-09T09:27:48.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.jar 2019-12-09T09:27:48.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.jar 2019-12-09T09:27:48.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.jar 2019-12-09T09:27:48.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.jar 2019-12-09T09:27:48.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.jar 2019-12-09T09:27:48.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.jar 2019-12-09T09:27:48.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.jar 2019-12-09T09:27:48.830 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.jar 2019-12-09T09:27:48.830 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.jar 2019-12-09T09:27:48.927 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.jar 2019-12-09T09:27:48.927 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.jar 2019-12-09T09:27:48.927 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.jar 2019-12-09T09:27:48.927 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.jar 2019-12-09T09:27:48.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.jar 2019-12-09T09:27:48.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.jar 2019-12-09T09:27:48.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.jar 2019-12-09T09:27:48.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.jar 2019-12-09T09:27:48.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.jar 2019-12-09T09:27:48.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.jar 2019-12-09T09:27:48.929 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.jar 2019-12-09T09:27:48.929 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.jar 2019-12-09T09:27:48.929 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.jar 2019-12-09T09:27:48.929 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.jar 2019-12-09T09:27:48.929 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.jar 2019-12-09T09:27:49.028 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.jar 2019-12-09T09:27:49.028 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.jar 2019-12-09T09:27:49.028 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.jar 2019-12-09T09:27:49.028 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.jar 2019-12-09T09:27:49.028 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.jar 2019-12-09T09:27:49.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.jar 2019-12-09T09:27:49.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.jar 2019-12-09T09:27:49.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.jar 2019-12-09T09:27:49.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.jar 2019-12-09T09:27:49.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.jar 2019-12-09T09:27:49.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.jar 2019-12-09T09:27:49.029 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.jar 2019-12-09T09:27:49.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.jar 2019-12-09T09:27:49.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.jar 2019-12-09T09:27:49.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.jar 2019-12-09T09:27:49.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.jar 2019-12-09T09:27:49.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.jar 2019-12-09T09:27:49.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.jar 2019-12-09T09:27:49.030 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.jar 2019-12-09T09:27:49.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.jar 2019-12-09T09:27:49.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.jar 2019-12-09T09:27:49.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.jar 2019-12-09T09:27:49.127 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.jar 2019-12-09T09:27:49.127 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.jar 2019-12-09T09:27:49.128 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.jar 2019-12-09T09:27:49.128 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.jar 2019-12-09T09:27:49.128 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.jar 2019-12-09T09:27:49.128 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.jar 2019-12-09T09:27:49.128 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.jar 2019-12-09T09:27:49.128 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.jar 2019-12-09T09:27:49.128 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.jar 2019-12-09T09:27:49.129 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.jar 2019-12-09T09:27:49.129 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.jar 2019-12-09T09:27:49.129 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.jar 2019-12-09T09:27:49.129 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.jar 2019-12-09T09:27:49.129 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.jar 2019-12-09T09:27:49.129 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.jar 2019-12-09T09:27:49.129 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.jar 2019-12-09T09:27:49.130 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.jar 2019-12-09T09:27:49.227 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.jar 2019-12-09T09:27:49.228 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.jar 2019-12-09T09:27:49.228 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.jar 2019-12-09T09:27:49.228 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.jar 2019-12-09T09:27:49.228 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.jar 2019-12-09T09:27:49.228 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.jar 2019-12-09T09:27:49.228 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.jar 2019-12-09T09:27:49.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.jar 2019-12-09T09:27:49.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.jar 2019-12-09T09:27:49.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.jar 2019-12-09T09:27:49.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.jar 2019-12-09T09:27:49.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.jar 2019-12-09T09:27:49.328 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.jar 2019-12-09T09:27:49.328 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.jar 2019-12-09T09:27:49.427 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.jar 2019-12-09T09:27:49.427 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.jar 2019-12-09T09:27:49.427 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.jar 2019-12-09T09:27:49.427 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.jar 2019-12-09T09:27:49.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.jar 2019-12-09T09:27:49.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.jar 2019-12-09T09:27:49.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.jar 2019-12-09T09:27:49.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.jar 2019-12-09T09:27:49.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.jar 2019-12-09T09:27:49.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.jar 2019-12-09T09:27:49.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.jar 2019-12-09T09:27:49.429 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.jar 2019-12-09T09:27:49.429 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.jar 2019-12-09T09:27:49.429 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.jar 2019-12-09T09:27:49.527 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.jar 2019-12-09T09:27:49.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.jar 2019-12-09T09:27:49.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.jar 2019-12-09T09:27:49.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.jar 2019-12-09T09:27:49.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.jar 2019-12-09T09:27:49.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.jar 2019-12-09T09:27:49.529 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.jar 2019-12-09T09:27:49.529 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.jar 2019-12-09T09:27:49.529 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.jar 2019-12-09T09:27:49.529 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.jar 2019-12-09T09:27:49.627 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.jar 2019-12-09T09:27:49.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.jar 2019-12-09T09:27:49.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.jar 2019-12-09T09:27:49.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.jar 2019-12-09T09:27:49.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.jar 2019-12-09T09:27:49.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.jar 2019-12-09T09:27:49.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.jar 2019-12-09T09:27:49.628 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.jar 2019-12-09T09:27:49.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.jar 2019-12-09T09:27:49.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.jar 2019-12-09T09:27:49.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.jar 2019-12-09T09:27:49.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.jar 2019-12-09T09:27:49.629 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.jar 2019-12-09T09:27:49.727 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.jar 2019-12-09T09:27:49.727 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.jar 2019-12-09T09:27:49.728 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.jar 2019-12-09T09:27:49.728 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.jar 2019-12-09T09:27:49.728 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.jar 2019-12-09T09:27:49.728 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.jar 2019-12-09T09:27:49.728 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.jar 2019-12-09T09:27:49.729 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.jar 2019-12-09T09:27:49.729 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.jar 2019-12-09T09:27:49.827 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.jar 2019-12-09T09:27:49.827 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.jar 2019-12-09T09:27:49.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.jar 2019-12-09T09:27:49.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.jar 2019-12-09T09:27:49.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.jar 2019-12-09T09:27:49.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.jar 2019-12-09T09:27:49.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.jar 2019-12-09T09:27:49.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.jar 2019-12-09T09:27:49.828 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.jar 2019-12-09T09:27:49.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.jar 2019-12-09T09:27:49.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.jar 2019-12-09T09:27:49.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.jar 2019-12-09T09:27:49.829 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.jar 2019-12-09T09:27:49.927 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.jar 2019-12-09T09:27:49.927 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.jar 2019-12-09T09:27:49.927 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.jar 2019-12-09T09:27:49.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.jar 2019-12-09T09:27:49.928 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.jar 2019-12-09T09:27:49.929 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.jar 2019-12-09T09:27:50.027 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.jar 2019-12-09T09:27:50.027 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.jar 2019-12-09T09:27:50.028 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.jar 2019-12-09T09:27:50.028 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.jar 2019-12-09T09:27:50.028 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.jar 2019-12-09T09:27:50.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.jar 2019-12-09T09:27:50.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.jar 2019-12-09T09:27:50.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.jar 2019-12-09T09:27:50.031 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.jar 2019-12-09T09:27:50.032 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.jar 2019-12-09T09:27:50.032 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.jar 2019-12-09T09:27:50.032 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.jar 2019-12-09T09:27:50.032 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.jar 2019-12-09T09:27:50.032 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.jar 2019-12-09T09:27:50.033 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.jar 2019-12-09T09:27:50.033 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.jar 2019-12-09T09:27:50.127 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.jar 2019-12-09T09:27:50.127 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.jar 2019-12-09T09:27:50.127 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.jar 2019-12-09T09:27:50.127 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.jar 2019-12-09T09:27:50.128 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.jar 2019-12-09T09:27:50.128 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.jar 2019-12-09T09:27:50.130 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.jar 2019-12-09T09:27:50.130 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.jar 2019-12-09T09:27:50.130 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.jar 2019-12-09T09:27:50.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.jar 2019-12-09T09:27:50.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.jar 2019-12-09T09:27:50.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.jar 2019-12-09T09:27:50.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.jar 2019-12-09T09:27:50.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.jar 2019-12-09T09:27:50.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.jar 2019-12-09T09:27:50.131 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.jar 2019-12-09T09:27:50.132 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.jar 2019-12-09T09:27:50.132 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.jar 2019-12-09T09:27:50.132 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.jar 2019-12-09T09:27:50.229 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar 2019-12-09T09:27:50.229 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.jar 2019-12-09T09:27:50.230 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar 2019-12-09T09:27:50.230 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.jar 2019-12-09T09:27:50.230 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.jar 2019-12-09T09:27:50.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar 2019-12-09T09:27:50.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.jar 2019-12-09T09:27:50.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.jar 2019-12-09T09:27:50.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.jar 2019-12-09T09:27:50.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.jar 2019-12-09T09:27:50.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.jar 2019-12-09T09:27:50.231 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.jar 2019-12-09T09:27:50.232 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.jar 2019-12-09T09:27:50.232 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.jar 2019-12-09T09:27:50.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.jar 2019-12-09T09:27:50.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.jar 2019-12-09T09:27:50.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.jar 2019-12-09T09:27:50.327 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.jar 2019-12-09T09:27:50.328 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.jar 2019-12-09T09:27:50.328 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.jar 2019-12-09T09:27:50.328 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar 2019-12-09T09:27:50.328 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.jar 2019-12-09T09:27:50.328 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.jar 2019-12-09T09:27:50.329 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.jar 2019-12-09T09:27:50.329 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.jar 2019-12-09T09:27:50.329 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.jar 2019-12-09T09:27:50.329 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.jar 2019-12-09T09:27:50.329 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.jar 2019-12-09T09:27:50.427 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar 2019-12-09T09:27:50.427 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar 2019-12-09T09:27:50.427 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.jar 2019-12-09T09:27:50.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.jar 2019-12-09T09:27:50.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.jar 2019-12-09T09:27:50.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.jar 2019-12-09T09:27:50.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.jar 2019-12-09T09:27:50.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.jar 2019-12-09T09:27:50.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.jar 2019-12-09T09:27:50.428 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.jar 2019-12-09T09:27:50.429 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.jar 2019-12-09T09:27:50.429 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.jar 2019-12-09T09:27:50.429 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar 2019-12-09T09:27:50.429 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar 2019-12-09T09:27:50.430 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.jar 2019-12-09T09:27:50.430 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.jar 2019-12-09T09:27:50.430 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.jar 2019-12-09T09:27:50.430 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.jar 2019-12-09T09:27:50.430 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.jar 2019-12-09T09:27:50.527 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.jar 2019-12-09T09:27:50.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar 2019-12-09T09:27:50.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.jar 2019-12-09T09:27:50.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.jar 2019-12-09T09:27:50.528 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.jar 2019-12-09T09:27:50.529 INFO:teuthology.orchestra.run.smithi035.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.jar 2019-12-09T09:27:59.527 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :compileJava 2019-12-09T09:27:59.528 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :processResources 2019-12-09T09:27:59.528 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :classes 2019-12-09T09:28:00.128 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:28:00.128 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :compileTestJava 2019-12-09T09:28:00.129 INFO:teuthology.orchestra.run.smithi035.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T09:28:00.129 INFO:teuthology.orchestra.run.smithi035.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T09:28:00.129 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:28:00.129 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T09:28:00.129 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :testClasses 2019-12-09T09:28:07.828 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:28:07.828 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :test 2019-12-09T09:28:07.828 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:28:07.828 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2019-12-09T09:28:49.196 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:16:25.474 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:28:49.208 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:16:25.486 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:28:49.249 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:16:25.502 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:28:49.251 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:16:25.504 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:28:49.268 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:16:25.522 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:28:49.270 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:16:25.524 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:28:49.437 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:16:25.690 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:28:49.459 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:16:25.736 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:28:49.473 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:16:25.726 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:29:07.832 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:29:07.833 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2019-12-09T09:29:12.733 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:29:12.733 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2019-12-09T09:29:15.533 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:29:15.533 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2019-12-09T09:29:22.633 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:29:22.634 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2019-12-09T09:29:43.835 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:29:43.835 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2019-12-09T09:29:51.536 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:29:51.536 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2019-12-09T09:29:52.135 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:29:52.136 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2019-12-09T09:29:53.636 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:29:53.636 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2019-12-09T09:29:55.736 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:29:55.736 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2019-12-09T09:29:58.019 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:17:34.297 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:29:58.059 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:17:34.338 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:29:58.113 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:17:34.361 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:29:58.115 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:17:34.393 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:29:58.142 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:17:34.390 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:29:58.176 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:17:34.424 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:29:58.181 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:17:34.429 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:29:58.192 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:17:34.440 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:29:58.246 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:17:34.494 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:30:01.536 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:30:01.536 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2019-12-09T09:30:45.340 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:30:45.341 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2019-12-09T09:31:01.941 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:31:01.941 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2019-12-09T09:34:00.207 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:21:36.488 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:34:00.212 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:21:36.493 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:34:00.225 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:21:36.455 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:34:00.265 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:21:36.494 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:34:00.299 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:21:36.528 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:34:00.300 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:21:36.530 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:34:00.389 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:21:36.671 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:34:00.490 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:21:36.719 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:34:00.522 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:21:36.752 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:34:58.459 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:34:58.459 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2019-12-09T09:35:11.060 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:35:11.060 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2019-12-09T09:35:13.260 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:35:13.260 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2019-12-09T09:35:17.660 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:35:17.661 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2019-12-09T09:35:22.360 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:35:22.361 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2019-12-09T09:35:32.561 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:35:32.561 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2019-12-09T09:36:05.261 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:36:05.262 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2019-12-09T09:36:11.662 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:36:11.662 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2019-12-09T09:36:11.862 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:36:11.862 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2019-12-09T09:36:12.561 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:36:12.562 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2019-12-09T09:36:33.562 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:36:33.562 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2019-12-09T09:36:41.865 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:36:41.865 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2019-12-09T09:37:07.762 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:37:07.762 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2019-12-09T09:37:25.461 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:37:25.462 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2019-12-09T09:37:49.261 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:37:49.264 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2019-12-09T09:37:57.461 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:37:57.462 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2019-12-09T09:38:20.108 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:25:56.388 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:38:20.115 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:25:56.391 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:38:20.127 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:25:56.350 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:38:20.134 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:25:56.356 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:38:20.138 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:25:56.361 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:38:20.156 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:25:56.378 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:38:20.253 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:25:56.534 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:38:20.269 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:25:56.492 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:38:20.329 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:25:56.552 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:44:24.269 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:32:00.536 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:44:24.301 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:32:00.509 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:44:24.301 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:32:00.517 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:44:24.302 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:32:00.511 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:44:24.302 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:32:00.516 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:44:24.302 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:32:00.552 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:44:24.533 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:32:00.756 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:44:24.540 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:32:00.808 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:44:24.540 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:32:00.763 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:47:42.161 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:47:42.186 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2019-12-09T09:47:51.161 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:47:51.161 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2019-12-09T09:48:38.861 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:48:38.862 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2019-12-09T09:48:46.261 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:48:46.262 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2019-12-09T09:49:25.761 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:49:25.761 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2019-12-09T09:49:34.161 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:49:34.162 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2019-12-09T09:49:36.581 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:37:12.845 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:49:36.589 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:37:12.853 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:49:36.592 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:37:12.815 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:49:36.597 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:37:12.820 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:49:36.600 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:37:12.823 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:49:36.600 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:37:12.823 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:49:36.799 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:37:13.021 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:49:36.831 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:37:13.095 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:49:36.847 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:37:13.070 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:49:41.461 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:49:41.462 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2019-12-09T09:49:54.061 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:49:54.061 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2019-12-09T09:49:56.761 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:49:56.762 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2019-12-09T09:50:00.761 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:50:00.761 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2019-12-09T09:50:04.861 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:50:04.862 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2019-12-09T09:50:11.661 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:50:11.661 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2019-12-09T09:50:14.561 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:50:14.562 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2019-12-09T09:50:20.161 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:50:20.161 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2019-12-09T09:50:24.361 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:50:24.362 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2019-12-09T09:50:33.661 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:50:33.724 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2019-12-09T09:50:38.561 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:50:38.561 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2019-12-09T09:50:46.661 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:50:46.661 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2019-12-09T09:50:49.862 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:50:49.862 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2019-12-09T09:50:58.161 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:50:58.162 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2019-12-09T09:51:00.361 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:51:00.361 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2019-12-09T09:51:06.161 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:51:06.162 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2019-12-09T09:51:35.864 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:51:35.864 INFO:teuthology.orchestra.run.smithi035.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2019-12-09T09:51:43.761 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:51:43.762 INFO:teuthology.orchestra.run.smithi035.stdout:---------------------------------------------------------------------- 2019-12-09T09:51:43.762 INFO:teuthology.orchestra.run.smithi035.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2019-12-09T09:51:43.762 INFO:teuthology.orchestra.run.smithi035.stdout:---------------------------------------------------------------------- 2019-12-09T09:51:43.762 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:51:43.762 INFO:teuthology.orchestra.run.smithi035.stdout:BUILD SUCCESSFUL in 24m 2s 2019-12-09T09:51:43.762 INFO:teuthology.orchestra.run.smithi035.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2019-12-09T09:51:44.088 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T09:51:44.223 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.221+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.244 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.241+0000 7ff338754ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:51:44.244 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.241+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.251 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.249+0000 7ff338754ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:51:44.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.249+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.256 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.254+0000 7ff338754ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:51:44.256 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.254+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.263 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.262+0000 7ff338754ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:51:44.447 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.444+0000 7ff338754ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:51:44.447 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.444+0000 7ff338754ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:51:44.448 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.444+0000 7ff338754ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:51:44.448 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.444+0000 7ff338754ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:51:44.448 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.444+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.464 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.457+0000 7ff338754ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:51:44.464 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.457+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.472+0000 7ff338754ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:51:44.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.473+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.486 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.484+0000 7ff338754ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:51:44.487 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.484+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.500 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.498+0000 7ff338754ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:51:44.500 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.498+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.507+0000 7ff338754ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:51:44.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.507+0000 7ff338754ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:51:44.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.507+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.516 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.514+0000 7ff338754ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:51:44.517 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.514+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.529 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.523+0000 7ff338754ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:51:44.529 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.523+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.536 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.534+0000 7ff338754ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:51:44.536 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.534+0000 7ff338754ec0 20 zonegroup default 2019-12-09T09:51:44.536 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.534+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.541 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.539+0000 7ff338754ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:51:44.541 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.539+0000 7ff338754ec0 10 Cannot find current period zone using local zone 2019-12-09T09:51:44.541 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.539+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.547 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.544+0000 7ff338754ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:51:44.547 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.544+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.555 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.553+0000 7ff338754ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:51:44.555 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.553+0000 7ff338754ec0 20 rados->read ofs=0 len=0 2019-12-09T09:51:44.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.562+0000 7ff338754ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:51:44.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.562+0000 7ff338754ec0 20 zone default found 2019-12-09T09:51:44.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.562+0000 7ff338754ec0 20 started sync module instance, tier type = 2019-12-09T09:51:44.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.562+0000 7ff338754ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:51:44.962 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.959+0000 7ff338754ec0 20 add_watcher() i=0 2019-12-09T09:51:44.962 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.959+0000 7ff338754ec0 20 add_watcher() i=1 2019-12-09T09:51:44.962 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.959+0000 7ff338754ec0 20 add_watcher() i=2 2019-12-09T09:51:44.963 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.959+0000 7ff338754ec0 20 add_watcher() i=3 2019-12-09T09:51:44.963 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.959+0000 7ff338754ec0 20 add_watcher() i=4 2019-12-09T09:51:44.963 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.959+0000 7ff338754ec0 20 add_watcher() i=5 2019-12-09T09:51:44.971 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:44.969+0000 7ff338754ec0 20 add_watcher() i=6 2019-12-09T09:51:45.005 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:45.003+0000 7ff338754ec0 20 add_watcher() i=7 2019-12-09T09:51:45.006 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:45.003+0000 7ff338754ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:51:45.006 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:45.003+0000 7ff2f67dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:51:45.015 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:45.012+0000 7ff32597e700 0 -- 172.21.15.35:0/235419550 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x555d331568e0 msgr2=0x555d3317bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T09:51:45.222 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:45.216+0000 7ff32597e700 0 --2- 172.21.15.35:0/235419550 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x555d331568e0 0x555d3317bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T09:51:45.271 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:45.269+0000 7ff32517d700 0 -- 172.21.15.35:0/192656385 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff268002800 msgr2=0x7ff268002d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:51:47.082 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.080+0000 7ff26cff9700 20 reqs_thread_entry: start 2019-12-09T09:51:47.083 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.080+0000 7ff2667fc700 20 reqs_thread_entry: start 2019-12-09T09:51:47.083 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.080+0000 7ff338754ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:51:47.083 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.081+0000 7ff264ff9700 20 reqs_thread_entry: start 2019-12-09T09:51:47.083 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.081+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:51:47.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.144+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:47.165 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.163+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:47.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.173+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:47.219 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.216+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:51:47.310 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.306+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:47.353 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.350+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:47.370 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.368+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:47.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.442+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:51:47.517 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.515+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:47.544 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.542+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:47.562 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.559+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:47.563 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.559+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.454', time=2019-12-09T11:34:02.137736+0000, chain.objs.size()=9 2019-12-09T09:51:47.563 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.559+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~dKB6RZitMqPO02Qk2GsfPQEvCyOl7Z-.1 2019-12-09T09:51:47.563 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.559+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~dKB6RZitMqPO02Qk2GsfPQEvCyOl7Z-.1_1 2019-12-09T09:51:47.563 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.559+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~dKB6RZitMqPO02Qk2GsfPQEvCyOl7Z-.2 2019-12-09T09:51:47.563 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.560+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~dKB6RZitMqPO02Qk2GsfPQEvCyOl7Z-.2_1 2019-12-09T09:51:47.563 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.560+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~dKB6RZitMqPO02Qk2GsfPQEvCyOl7Z-.3 2019-12-09T09:51:47.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.560+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~dKB6RZitMqPO02Qk2GsfPQEvCyOl7Z-.3_1 2019-12-09T09:51:47.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.560+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~dKB6RZitMqPO02Qk2GsfPQEvCyOl7Z-.4 2019-12-09T09:51:47.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.560+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~dKB6RZitMqPO02Qk2GsfPQEvCyOl7Z-.4_1 2019-12-09T09:51:47.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.560+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~dKB6RZitMqPO02Qk2GsfPQEvCyOl7Z-.5 2019-12-09T09:51:47.817 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.814+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:47.896 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.894+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:51:47.942 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.940+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:47.962 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.959+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:47.976 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.973+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:47.976 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.973+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.466', time=2019-12-09T11:34:57.179126+0000, chain.objs.size()=7 2019-12-09T09:51:47.977 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.973+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~HXQZpCnnRUPqN1_uJ10duC1zj51Z6db.1 2019-12-09T09:51:47.977 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.973+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~HXQZpCnnRUPqN1_uJ10duC1zj51Z6db.1_1 2019-12-09T09:51:47.977 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.973+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~HXQZpCnnRUPqN1_uJ10duC1zj51Z6db.1_2 2019-12-09T09:51:47.977 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.973+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~HXQZpCnnRUPqN1_uJ10duC1zj51Z6db.2 2019-12-09T09:51:47.977 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.973+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~HXQZpCnnRUPqN1_uJ10duC1zj51Z6db.2_1 2019-12-09T09:51:47.978 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.973+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~HXQZpCnnRUPqN1_uJ10duC1zj51Z6db.2_2 2019-12-09T09:51:47.978 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:47.973+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~HXQZpCnnRUPqN1_uJ10duC1zj51Z6db.3 2019-12-09T09:51:48.184 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.182+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:48.304 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.301+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:51:48.378 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.376+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:48.403 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.401+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:48.455 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.452+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:48.455 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.452+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.417', time=2019-12-09T11:30:47.502066+0000, chain.objs.size()=9 2019-12-09T09:51:48.455 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.452+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__multipart_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.1 2019-12-09T09:51:48.455 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.452+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__shadow_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.1_1 2019-12-09T09:51:48.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.452+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__multipart_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.2 2019-12-09T09:51:48.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.452+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__shadow_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.2_1 2019-12-09T09:51:48.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.452+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__multipart_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.3 2019-12-09T09:51:48.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.452+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__shadow_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.3_1 2019-12-09T09:51:48.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.452+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__multipart_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.4 2019-12-09T09:51:48.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.452+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__shadow_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.4_1 2019-12-09T09:51:48.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.452+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__multipart_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.5 2019-12-09T09:51:48.661 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.659+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:48.875 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.870+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:51:48.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.941+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:48.964 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.962+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:48.985 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.983+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:48.985 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.983+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.430', time=2019-12-09T11:35:06.314209+0000, chain.objs.size()=5 2019-12-09T09:51:48.985 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.983+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.19__shadow_.qy58nUwhhc9V7nUGYIaVu-RJY51JXgg_1 2019-12-09T09:51:48.985 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.983+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.19__shadow_.qy58nUwhhc9V7nUGYIaVu-RJY51JXgg_2 2019-12-09T09:51:48.985 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.983+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.19__shadow_.qy58nUwhhc9V7nUGYIaVu-RJY51JXgg_3 2019-12-09T09:51:48.986 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.983+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.19__shadow_.qy58nUwhhc9V7nUGYIaVu-RJY51JXgg_4 2019-12-09T09:51:48.986 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:48.983+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.19__shadow_.qy58nUwhhc9V7nUGYIaVu-RJY51JXgg_5 2019-12-09T09:51:49.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.173+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:49.336 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.333+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:51:49.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.418+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:49.442 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.439+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:49.460 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.457+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:49.460 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.457+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.659', time=2019-12-09T11:48:40.759006+0000, chain.objs.size()=9 2019-12-09T09:51:49.460 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.457+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.29__multipart_key1.2~XqsH1X3HjPzKpKzY_0rJtElnktDoy5J.1 2019-12-09T09:51:49.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.457+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.29__shadow_key1.2~XqsH1X3HjPzKpKzY_0rJtElnktDoy5J.1_1 2019-12-09T09:51:49.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.457+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.29__multipart_key1.2~XqsH1X3HjPzKpKzY_0rJtElnktDoy5J.2 2019-12-09T09:51:49.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.457+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.29__shadow_key1.2~XqsH1X3HjPzKpKzY_0rJtElnktDoy5J.2_1 2019-12-09T09:51:49.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.457+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.29__multipart_key1.2~XqsH1X3HjPzKpKzY_0rJtElnktDoy5J.3 2019-12-09T09:51:49.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.457+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.29__shadow_key1.2~XqsH1X3HjPzKpKzY_0rJtElnktDoy5J.3_1 2019-12-09T09:51:49.462 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.457+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.29__multipart_key1.2~XqsH1X3HjPzKpKzY_0rJtElnktDoy5J.4 2019-12-09T09:51:49.462 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.457+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.29__shadow_key1.2~XqsH1X3HjPzKpKzY_0rJtElnktDoy5J.4_1 2019-12-09T09:51:49.462 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.457+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.29__multipart_key1.2~XqsH1X3HjPzKpKzY_0rJtElnktDoy5J.5 2019-12-09T09:51:49.793 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.791+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:49.895 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.893+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:51:49.948 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.946+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:49.971 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.969+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:49.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T09:51:49.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.559', time=2019-12-09T11:37:10.613288+0000, chain.objs.size()=9 2019-12-09T09:51:49.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__multipart_file.mpg.2~619u2z6qq9XcsYe-rHjacz4pojTLTIp.1 2019-12-09T09:51:49.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__shadow_file.mpg.2~619u2z6qq9XcsYe-rHjacz4pojTLTIp.1_1 2019-12-09T09:51:49.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__multipart_file.mpg.2~619u2z6qq9XcsYe-rHjacz4pojTLTIp.2 2019-12-09T09:51:49.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__shadow_file.mpg.2~619u2z6qq9XcsYe-rHjacz4pojTLTIp.2_1 2019-12-09T09:51:50.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__multipart_file.mpg.2~619u2z6qq9XcsYe-rHjacz4pojTLTIp.3 2019-12-09T09:51:50.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__shadow_file.mpg.2~619u2z6qq9XcsYe-rHjacz4pojTLTIp.3_1 2019-12-09T09:51:50.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__multipart_file.mpg.2~619u2z6qq9XcsYe-rHjacz4pojTLTIp.4 2019-12-09T09:51:50.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__shadow_file.mpg.2~619u2z6qq9XcsYe-rHjacz4pojTLTIp.4_1 2019-12-09T09:51:50.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__multipart_file.mpg.2~619u2z6qq9XcsYe-rHjacz4pojTLTIp.5 2019-12-09T09:51:50.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.560', time=2019-12-09T11:37:13.651524+0000, chain.objs.size()=9 2019-12-09T09:51:50.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__multipart_file2.mpg.2~MHsNSo4tD481CMJJKQNcz9jEJchO_Gz.1 2019-12-09T09:51:50.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__shadow_file2.mpg.2~MHsNSo4tD481CMJJKQNcz9jEJchO_Gz.1_1 2019-12-09T09:51:50.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:49.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__multipart_file2.mpg.2~MHsNSo4tD481CMJJKQNcz9jEJchO_Gz.2 2019-12-09T09:51:50.200 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.198+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__shadow_file2.mpg.2~MHsNSo4tD481CMJJKQNcz9jEJchO_Gz.2_1 2019-12-09T09:51:50.201 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.198+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__multipart_file2.mpg.2~MHsNSo4tD481CMJJKQNcz9jEJchO_Gz.3 2019-12-09T09:51:50.201 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.198+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__shadow_file2.mpg.2~MHsNSo4tD481CMJJKQNcz9jEJchO_Gz.3_1 2019-12-09T09:51:50.376 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__multipart_file2.mpg.2~MHsNSo4tD481CMJJKQNcz9jEJchO_Gz.4 2019-12-09T09:51:50.376 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__shadow_file2.mpg.2~MHsNSo4tD481CMJJKQNcz9jEJchO_Gz.4_1 2019-12-09T09:51:50.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.26__multipart_file2.mpg.2~MHsNSo4tD481CMJJKQNcz9jEJchO_Gz.5 2019-12-09T09:51:50.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.374+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.608', time=2019-12-09T11:42:34.650589+0000, chain.objs.size()=21 2019-12-09T09:51:50.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.1 2019-12-09T09:51:50.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.1_1 2019-12-09T09:51:50.378 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.2 2019-12-09T09:51:50.378 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.2_1 2019-12-09T09:51:50.378 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.3 2019-12-09T09:51:50.378 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.3_1 2019-12-09T09:51:50.394 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.392+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.4 2019-12-09T09:51:50.484 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.482+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.4_1 2019-12-09T09:51:50.589 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.586+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.5 2019-12-09T09:51:50.589 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.587+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.5_1 2019-12-09T09:51:50.589 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.587+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.6 2019-12-09T09:51:50.597 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.594+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.6_1 2019-12-09T09:51:50.680 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.678+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.7 2019-12-09T09:51:50.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.678+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.7_1 2019-12-09T09:51:50.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.678+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.8 2019-12-09T09:51:50.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.678+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.8_1 2019-12-09T09:51:50.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.678+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.9 2019-12-09T09:51:50.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.678+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.9_1 2019-12-09T09:51:50.682 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.678+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.10 2019-12-09T09:51:50.682 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.678+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.10_1 2019-12-09T09:51:50.682 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.678+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~224k17cKm0OLZOD3ygtbVxwVR1ixBAN.11 2019-12-09T09:51:50.920 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.918+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:51.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:50.999+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:51:51.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.108+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:51.154 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.152+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:51.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:51.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.838', time=2019-12-09T11:51:37.863985+0000, chain.objs.size()=21 2019-12-09T09:51:51.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__multipart_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.1 2019-12-09T09:51:51.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__shadow_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.1_1 2019-12-09T09:51:51.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__multipart_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.2 2019-12-09T09:51:51.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__shadow_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.2_1 2019-12-09T09:51:51.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__multipart_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.3 2019-12-09T09:51:51.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__shadow_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.3_1 2019-12-09T09:51:51.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__multipart_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.4 2019-12-09T09:51:51.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__shadow_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.4_1 2019-12-09T09:51:51.176 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__multipart_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.5 2019-12-09T09:51:51.176 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__shadow_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.5_1 2019-12-09T09:51:51.176 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__multipart_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.6 2019-12-09T09:51:51.177 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.171+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__shadow_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.6_1 2019-12-09T09:51:51.319 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.317+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__multipart_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.7 2019-12-09T09:51:51.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.320+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__shadow_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.7_1 2019-12-09T09:51:51.419 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.416+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__multipart_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.8 2019-12-09T09:51:51.419 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.416+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__shadow_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.8_1 2019-12-09T09:51:51.419 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.416+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__multipart_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.9 2019-12-09T09:51:51.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.419+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__shadow_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.9_1 2019-12-09T09:51:51.531 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.528+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__multipart_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.10 2019-12-09T09:51:51.531 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.529+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__shadow_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.10_1 2019-12-09T09:51:51.531 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.529+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.39__multipart_key1.2~aff1PoE_mI2GEIM9f4lU-RXbekJ_MwH.11 2019-12-09T09:51:51.691 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.689+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:51.794 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.792+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:51:51.846 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.844+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:51.862 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.860+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:51.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:51:51.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.415', time=2019-12-09T11:30:57.201383+0000, chain.objs.size()=9 2019-12-09T09:51:51.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__multipart_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.1 2019-12-09T09:51:51.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__shadow_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.1_1 2019-12-09T09:51:51.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__multipart_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.2 2019-12-09T09:51:51.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__shadow_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.2_1 2019-12-09T09:51:51.880 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__multipart_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.3 2019-12-09T09:51:51.880 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__shadow_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.3_1 2019-12-09T09:51:51.880 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__multipart_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.4 2019-12-09T09:51:51.880 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__shadow_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.4_1 2019-12-09T09:51:51.880 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.17__multipart_key1.2~6YaeGG4bKoMhYTWhGUDrgsrib5rDK6B.5 2019-12-09T09:51:51.880 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.472', time=2019-12-09T11:35:01.253905+0000, chain.objs.size()=7 2019-12-09T09:51:51.881 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~2Qs_9C5_qREpFChd0Jj_hNoh1uRAxmA.1 2019-12-09T09:51:51.881 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~2Qs_9C5_qREpFChd0Jj_hNoh1uRAxmA.1_1 2019-12-09T09:51:51.881 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:51.876+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~2Qs_9C5_qREpFChd0Jj_hNoh1uRAxmA.1_2 2019-12-09T09:51:52.151 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.149+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~2Qs_9C5_qREpFChd0Jj_hNoh1uRAxmA.2 2019-12-09T09:51:52.151 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.149+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~2Qs_9C5_qREpFChd0Jj_hNoh1uRAxmA.2_1 2019-12-09T09:51:52.151 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.149+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~2Qs_9C5_qREpFChd0Jj_hNoh1uRAxmA.2_2 2019-12-09T09:51:52.203 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.201+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~2Qs_9C5_qREpFChd0Jj_hNoh1uRAxmA.3 2019-12-09T09:51:52.320 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.318+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:52.403 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.401+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:51:52.473 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.461+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:52.481 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.479+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:52.489 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.487+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:52.530 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.526+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:51:52.570 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.568+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:52.590 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.586+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:52.606 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.603+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:52.607 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.603+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.386', time=2019-12-09T11:29:45.090971+0000, chain.objs.size()=9 2019-12-09T09:51:52.607 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.603+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.15__multipart_key1.2~fOKnqwf6PZmzAab8_FhA4t32N7UsLGM.1 2019-12-09T09:51:52.607 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.603+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.15__shadow_key1.2~fOKnqwf6PZmzAab8_FhA4t32N7UsLGM.1_1 2019-12-09T09:51:52.607 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.603+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.15__multipart_key1.2~fOKnqwf6PZmzAab8_FhA4t32N7UsLGM.2 2019-12-09T09:51:52.607 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.603+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.15__shadow_key1.2~fOKnqwf6PZmzAab8_FhA4t32N7UsLGM.2_1 2019-12-09T09:51:52.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.603+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.15__multipart_key1.2~fOKnqwf6PZmzAab8_FhA4t32N7UsLGM.3 2019-12-09T09:51:52.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.603+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.15__shadow_key1.2~fOKnqwf6PZmzAab8_FhA4t32N7UsLGM.3_1 2019-12-09T09:51:52.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.604+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.15__multipart_key1.2~fOKnqwf6PZmzAab8_FhA4t32N7UsLGM.4 2019-12-09T09:51:52.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.604+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.15__shadow_key1.2~fOKnqwf6PZmzAab8_FhA4t32N7UsLGM.4_1 2019-12-09T09:51:52.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.604+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.15__multipart_key1.2~fOKnqwf6PZmzAab8_FhA4t32N7UsLGM.5 2019-12-09T09:51:52.824 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.822+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:52.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.942+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:51:52.983 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.981+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:53.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:52.999+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:53.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.008+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:53.059 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.055+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:51:53.107 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.104+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:53.128 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.126+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:53.143 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.140+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:53.143 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.140+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.460', time=2019-12-09T11:34:29.455073+0000, chain.objs.size()=7 2019-12-09T09:51:53.144 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.140+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~MwYAhYfKdv6bBD-ibaFXJeXODfm028n.1 2019-12-09T09:51:53.144 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.140+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~MwYAhYfKdv6bBD-ibaFXJeXODfm028n.1_1 2019-12-09T09:51:53.144 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.140+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~MwYAhYfKdv6bBD-ibaFXJeXODfm028n.1_2 2019-12-09T09:51:53.144 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.140+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~MwYAhYfKdv6bBD-ibaFXJeXODfm028n.2 2019-12-09T09:51:53.144 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.140+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~MwYAhYfKdv6bBD-ibaFXJeXODfm028n.2_1 2019-12-09T09:51:53.145 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.140+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~MwYAhYfKdv6bBD-ibaFXJeXODfm028n.2_2 2019-12-09T09:51:53.145 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.140+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~MwYAhYfKdv6bBD-ibaFXJeXODfm028n.3 2019-12-09T09:51:53.425 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.423+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:53.502 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.500+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:51:53.547 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.545+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:53.566 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.564+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:53.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:53.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.628', time=2019-12-09T11:46:20.181647+0000, chain.objs.size()=16 2019-12-09T09:51:53.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.1 2019-12-09T09:51:53.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.1_1 2019-12-09T09:51:53.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.1_2 2019-12-09T09:51:53.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.2 2019-12-09T09:51:53.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.2_1 2019-12-09T09:51:53.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.2_2 2019-12-09T09:51:53.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.3 2019-12-09T09:51:53.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.3_1 2019-12-09T09:51:53.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.3_2 2019-12-09T09:51:53.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.4 2019-12-09T09:51:53.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.4_1 2019-12-09T09:51:53.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.605+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.4_2 2019-12-09T09:51:53.797 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.795+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.5 2019-12-09T09:51:53.797 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.795+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.5_1 2019-12-09T09:51:53.798 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.795+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.5_2 2019-12-09T09:51:53.798 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.795+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~wjxouZfu1WXdEqV_qzp1TgT454qevpc.6 2019-12-09T09:51:53.971 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:53.969+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:54.068 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.066+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:51:54.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.114+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:54.133 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.131+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:54.145 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.143+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:54.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.143+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~mXYlB34W-mJC0GcHsHounAVmkO5KqUW', time=2019-12-09T11:37:52.526218+0000, chain.objs.size()=5 2019-12-09T09:51:54.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.143+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.27__multipart_key1.2~mXYlB34W-mJC0GcHsHounAVmkO5KqUW.7 2019-12-09T09:51:54.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.143+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.27__shadow_key1.2~mXYlB34W-mJC0GcHsHounAVmkO5KqUW.7_1 2019-12-09T09:51:54.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.143+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.27__multipart_key1.2~mXYlB34W-mJC0GcHsHounAVmkO5KqUW.10 2019-12-09T09:51:54.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.143+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.27__shadow_key1.2~mXYlB34W-mJC0GcHsHounAVmkO5KqUW.10_1 2019-12-09T09:51:54.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.143+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.27__multipart_key1.2~mXYlB34W-mJC0GcHsHounAVmkO5KqUW.13 2019-12-09T09:51:54.336 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.334+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:54.443 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.441+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:51:54.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.488+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:54.505 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.503+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:54.515 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.512+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:54.553 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.551+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:51:54.613 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.609+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:54.629 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.627+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:54.641 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.639+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:54.676 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.673+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:51:54.738 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.735+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:54.771 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.769+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:54.787 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.784+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:54.822 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.820+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:51:54.862 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.860+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:54.880 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.876+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:54.887 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.885+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:54.916 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:54.914+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:51:55.018 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.015+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:55.051 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.049+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:55.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:51:55.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.528', time=2019-12-09T11:36:37.385651+0000, chain.objs.size()=9 2019-12-09T09:51:55.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.25__multipart_key1.2~mlnbZ3Si2TiaXeNG9D9aGSeHcsPYVWA.1 2019-12-09T09:51:55.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.25__shadow_key1.2~mlnbZ3Si2TiaXeNG9D9aGSeHcsPYVWA.1_1 2019-12-09T09:51:55.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.25__multipart_key1.2~mlnbZ3Si2TiaXeNG9D9aGSeHcsPYVWA.2 2019-12-09T09:51:55.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.25__shadow_key1.2~mlnbZ3Si2TiaXeNG9D9aGSeHcsPYVWA.2_1 2019-12-09T09:51:55.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.25__multipart_key1.2~mlnbZ3Si2TiaXeNG9D9aGSeHcsPYVWA.3 2019-12-09T09:51:55.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.25__shadow_key1.2~mlnbZ3Si2TiaXeNG9D9aGSeHcsPYVWA.3_1 2019-12-09T09:51:55.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.25__multipart_key1.2~mlnbZ3Si2TiaXeNG9D9aGSeHcsPYVWA.4 2019-12-09T09:51:55.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.25__shadow_key1.2~mlnbZ3Si2TiaXeNG9D9aGSeHcsPYVWA.4_1 2019-12-09T09:51:55.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.25__multipart_key1.2~mlnbZ3Si2TiaXeNG9D9aGSeHcsPYVWA.5 2019-12-09T09:51:55.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.636', time=2019-12-09T11:47:40.680236+0000, chain.objs.size()=16 2019-12-09T09:51:55.093 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.1 2019-12-09T09:51:55.093 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.1_1 2019-12-09T09:51:55.093 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.088+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.1_2 2019-12-09T09:51:55.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.305+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.2 2019-12-09T09:51:55.376 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.2_1 2019-12-09T09:51:55.376 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.2_2 2019-12-09T09:51:55.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.3 2019-12-09T09:51:55.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.3_1 2019-12-09T09:51:55.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.3_2 2019-12-09T09:51:55.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.4 2019-12-09T09:51:55.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.4_1 2019-12-09T09:51:55.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.4_2 2019-12-09T09:51:55.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.5 2019-12-09T09:51:55.378 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.374+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.5_1 2019-12-09T09:51:55.536 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.534+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.5_2 2019-12-09T09:51:55.597 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.595+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~YG9gvVAuegQf7k2ApBN2izjx0fCCklX.6 2019-12-09T09:51:55.741 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.736+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:55.820 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.818+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:51:55.864 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.862+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:55.881 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.879+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:55.919 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T09:51:55.919 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.595', time=2019-12-09T11:40:59.770415+0000, chain.objs.size()=21 2019-12-09T09:51:55.919 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.1 2019-12-09T09:51:55.920 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.1_1 2019-12-09T09:51:55.920 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.2 2019-12-09T09:51:55.920 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.2_1 2019-12-09T09:51:55.920 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.3 2019-12-09T09:51:55.920 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.3_1 2019-12-09T09:51:55.921 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.4 2019-12-09T09:51:55.921 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.4_1 2019-12-09T09:51:55.921 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.5 2019-12-09T09:51:55.921 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.5_1 2019-12-09T09:51:55.921 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.6 2019-12-09T09:51:55.922 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:55.916+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.6_1 2019-12-09T09:51:56.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.109+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.7 2019-12-09T09:51:56.147 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.144+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.7_1 2019-12-09T09:51:56.147 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.144+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.8 2019-12-09T09:51:56.147 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.145+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.8_1 2019-12-09T09:51:56.148 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.145+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.9 2019-12-09T09:51:56.148 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.145+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.9_1 2019-12-09T09:51:56.176 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.174+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.10 2019-12-09T09:51:56.177 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.174+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.10_1 2019-12-09T09:51:56.177 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.174+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~aFlVKMbfgnUOae2s8FfdbGEhr9wNsjo.11 2019-12-09T09:51:56.177 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.174+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.644', time=2019-12-09T11:47:44.612695+0000, chain.objs.size()=16 2019-12-09T09:51:56.177 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.174+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.1 2019-12-09T09:51:56.177 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.174+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.1_1 2019-12-09T09:51:56.335 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.332+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.1_2 2019-12-09T09:51:56.365 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.363+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.2 2019-12-09T09:51:56.384 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.382+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.2_1 2019-12-09T09:51:56.384 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.382+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.2_2 2019-12-09T09:51:56.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.382+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.3 2019-12-09T09:51:56.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.382+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.3_1 2019-12-09T09:51:56.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.382+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.3_2 2019-12-09T09:51:56.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.382+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.4 2019-12-09T09:51:56.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.382+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.4_1 2019-12-09T09:51:56.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.382+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.4_2 2019-12-09T09:51:56.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.382+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.5 2019-12-09T09:51:56.707 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.704+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.5_1 2019-12-09T09:51:56.707 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.705+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.5_2 2019-12-09T09:51:56.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.705+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~0jAfTowurMnrWwptHXjWfXpXDW9pW4c.6 2019-12-09T09:51:56.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.705+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O', time=2019-12-09T11:49:48.482493+0000, chain.objs.size()=30 2019-12-09T09:51:56.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.705+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.1 2019-12-09T09:51:56.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.705+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.2 2019-12-09T09:51:56.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.705+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.3 2019-12-09T09:51:56.709 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.705+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.4 2019-12-09T09:51:56.709 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.705+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.5 2019-12-09T09:51:56.709 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.705+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.6 2019-12-09T09:51:56.710 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.705+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.7 2019-12-09T09:51:56.710 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.705+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.8 2019-12-09T09:51:56.882 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.879+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.9 2019-12-09T09:51:56.890 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.888+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.10 2019-12-09T09:51:56.891 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.888+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.11 2019-12-09T09:51:56.891 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.888+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.12 2019-12-09T09:51:56.891 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.888+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.13 2019-12-09T09:51:56.891 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.888+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.Lj5qnPueaIJnryqIA3rynfy91ZxKG5x.14 2019-12-09T09:51:56.891 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.888+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.VRDNZa9cGQUv0vSaKsZkliFdzljNAeF.15 2019-12-09T09:51:56.980 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.978+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.UeJoBtQgt0KSE36SPJMgND5h3wG4Zaq.16 2019-12-09T09:51:56.981 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.978+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.SD1sApSqO1IxoLfYasgvwdChRWK1a5_.17 2019-12-09T09:51:56.981 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.978+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.MdLoDepnrPMEvGfu6bjVifm7dJ-PCqo.18 2019-12-09T09:51:56.981 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:56.978+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.0Fd9LSa_bh1ZtAgHiOJcN-uP7CnhskT.19 2019-12-09T09:51:57.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.009+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.gytbTTYwAxJJdZ_RBwJ5cFvf8X2pfMt.20 2019-12-09T09:51:57.058 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.056+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.21 2019-12-09T09:51:57.087 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.085+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.22 2019-12-09T09:51:57.162 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.159+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.23 2019-12-09T09:51:57.162 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.160+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.24 2019-12-09T09:51:57.162 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.160+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.25 2019-12-09T09:51:57.162 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.160+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.26 2019-12-09T09:51:57.162 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.160+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.27 2019-12-09T09:51:57.163 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.160+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.28 2019-12-09T09:51:57.172 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.169+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.29 2019-12-09T09:51:57.172 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.169+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.31__multipart_key1.2~hqIJAMb0RctFMmDLtwFUK4hSLfpMJ3O.30 2019-12-09T09:51:57.482 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.480+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:57.591 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.589+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:51:57.654 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.652+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:57.682 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.680+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:57.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.691+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:57.752 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.750+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:51:57.811 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.809+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:57.843 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.841+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:57.865 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.863+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:57.900 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.898+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:51:57.957 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.955+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:57.977 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.975+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:57.998 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:57.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.620', time=2019-12-09T11:44:28.111389+0000, chain.objs.size()=19 2019-12-09T09:51:57.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.1 2019-12-09T09:51:57.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.1_1 2019-12-09T09:51:57.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.2 2019-12-09T09:51:57.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.2_1 2019-12-09T09:51:57.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.3 2019-12-09T09:51:58.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.3_1 2019-12-09T09:51:58.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.4 2019-12-09T09:51:58.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.4_1 2019-12-09T09:51:58.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.5 2019-12-09T09:51:58.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.5_1 2019-12-09T09:51:58.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.6 2019-12-09T09:51:58.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:57.996+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.6_1 2019-12-09T09:51:58.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.115+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.7 2019-12-09T09:51:58.202 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.200+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.7_1 2019-12-09T09:51:58.203 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.200+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.8 2019-12-09T09:51:58.243 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.241+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.8_1 2019-12-09T09:51:58.244 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.241+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.9 2019-12-09T09:51:58.244 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.241+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__shadow_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.9_1 2019-12-09T09:51:58.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.281+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.28__multipart_key1.2~pnTa9w_JQUJit2HMDwrAWhgQfqEa6Wa.10 2019-12-09T09:51:58.427 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.425+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:58.545 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.543+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:51:58.591 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.589+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:58.632 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.628+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:58.640 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.638+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:58.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.682+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:51:58.731 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.729+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:58.748 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.746+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:58.756 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.754+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:58.843 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.841+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:51:58.905 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.902+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:58.935 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.932+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:58.954 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:51:58.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~ysc0g-vdhZ6a70MbHrmtYjUDnCTZLla', time=2019-12-09T11:29:06.632461+0000, chain.objs.size()=9 2019-12-09T09:51:58.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.13__multipart_key1.2~ysc0g-vdhZ6a70MbHrmtYjUDnCTZLla.1 2019-12-09T09:51:58.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.13__shadow_key1.2~ysc0g-vdhZ6a70MbHrmtYjUDnCTZLla.1_1 2019-12-09T09:51:58.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.13__multipart_key1.2~ysc0g-vdhZ6a70MbHrmtYjUDnCTZLla.2 2019-12-09T09:51:58.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.13__shadow_key1.2~ysc0g-vdhZ6a70MbHrmtYjUDnCTZLla.2_1 2019-12-09T09:51:58.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.13__multipart_key1.2~ysc0g-vdhZ6a70MbHrmtYjUDnCTZLla.3 2019-12-09T09:51:58.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.13__shadow_key1.2~ysc0g-vdhZ6a70MbHrmtYjUDnCTZLla.3_1 2019-12-09T09:51:58.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.13__multipart_key1.2~ysc0g-vdhZ6a70MbHrmtYjUDnCTZLla.4 2019-12-09T09:51:58.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.13__shadow_key1.2~ysc0g-vdhZ6a70MbHrmtYjUDnCTZLla.4_1 2019-12-09T09:51:58.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.13__multipart_key1.2~ysc0g-vdhZ6a70MbHrmtYjUDnCTZLla.5 2019-12-09T09:51:58.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.438', time=2019-12-09T11:33:01.563641+0000, chain.objs.size()=9 2019-12-09T09:51:58.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~NvI7OBD99JO2uz9G-YDLLWMDBcaETtm.1 2019-12-09T09:51:58.957 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~NvI7OBD99JO2uz9G-YDLLWMDBcaETtm.1_1 2019-12-09T09:51:58.957 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:58.952+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~NvI7OBD99JO2uz9G-YDLLWMDBcaETtm.2 2019-12-09T09:51:59.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.172+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~NvI7OBD99JO2uz9G-YDLLWMDBcaETtm.2_1 2019-12-09T09:51:59.237 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.235+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~NvI7OBD99JO2uz9G-YDLLWMDBcaETtm.3 2019-12-09T09:51:59.237 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.235+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~NvI7OBD99JO2uz9G-YDLLWMDBcaETtm.3_1 2019-12-09T09:51:59.275 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.272+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~NvI7OBD99JO2uz9G-YDLLWMDBcaETtm.4 2019-12-09T09:51:59.275 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.273+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~NvI7OBD99JO2uz9G-YDLLWMDBcaETtm.4_1 2019-12-09T09:51:59.275 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.273+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~NvI7OBD99JO2uz9G-YDLLWMDBcaETtm.5 2019-12-09T09:51:59.489 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.487+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:51:59.593 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.591+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:51:59.635 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.633+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:51:59.651 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.648+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:51:59.664 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.662+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:51:59.664 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.662+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.446', time=2019-12-09T11:33:36.325094+0000, chain.objs.size()=9 2019-12-09T09:51:59.665 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.662+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~mmfFw2gOXsleOBzGpdj4j5TfAtaOKnV.1 2019-12-09T09:51:59.665 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.662+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~mmfFw2gOXsleOBzGpdj4j5TfAtaOKnV.1_1 2019-12-09T09:51:59.665 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.662+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~mmfFw2gOXsleOBzGpdj4j5TfAtaOKnV.2 2019-12-09T09:51:59.665 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.662+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~mmfFw2gOXsleOBzGpdj4j5TfAtaOKnV.2_1 2019-12-09T09:51:59.665 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.662+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~mmfFw2gOXsleOBzGpdj4j5TfAtaOKnV.3 2019-12-09T09:51:59.665 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.662+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~mmfFw2gOXsleOBzGpdj4j5TfAtaOKnV.3_1 2019-12-09T09:51:59.665 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.662+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~mmfFw2gOXsleOBzGpdj4j5TfAtaOKnV.4 2019-12-09T09:51:59.666 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.662+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__shadow_key1.2~mmfFw2gOXsleOBzGpdj4j5TfAtaOKnV.4_1 2019-12-09T09:51:59.666 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.662+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.20__multipart_key1.2~mmfFw2gOXsleOBzGpdj4j5TfAtaOKnV.5 2019-12-09T09:51:59.911 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:51:59.908+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:52:00.007 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.005+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:52:00.058 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.056+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:00.084 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.082+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:00.101 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:52:00.101 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.509', time=2019-12-09T11:36:06.442874+0000, chain.objs.size()=9 2019-12-09T09:52:00.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.24__multipart_key1.2~S5BZL3OwYalarP4jTTy3JNGhW32sQDx.1 2019-12-09T09:52:00.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.24__shadow_key1.2~S5BZL3OwYalarP4jTTy3JNGhW32sQDx.1_1 2019-12-09T09:52:00.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.24__multipart_key1.2~S5BZL3OwYalarP4jTTy3JNGhW32sQDx.2 2019-12-09T09:52:00.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.24__shadow_key1.2~S5BZL3OwYalarP4jTTy3JNGhW32sQDx.2_1 2019-12-09T09:52:00.103 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.24__multipart_key1.2~S5BZL3OwYalarP4jTTy3JNGhW32sQDx.3 2019-12-09T09:52:00.103 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.24__shadow_key1.2~S5BZL3OwYalarP4jTTy3JNGhW32sQDx.3_1 2019-12-09T09:52:00.103 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.24__multipart_key1.2~S5BZL3OwYalarP4jTTy3JNGhW32sQDx.4 2019-12-09T09:52:00.103 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.24__shadow_key1.2~S5BZL3OwYalarP4jTTy3JNGhW32sQDx.4_1 2019-12-09T09:52:00.104 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.24__multipart_key1.2~S5BZL3OwYalarP4jTTy3JNGhW32sQDx.5 2019-12-09T09:52:00.104 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.674', time=2019-12-09T11:49:28.743212+0000, chain.objs.size()=9 2019-12-09T09:52:00.104 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.30__multipart_key1.2~Y1Ohv4hw_F75Gys1zjy_NPAC3dli1oJ.1 2019-12-09T09:52:00.105 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.30__shadow_key1.2~Y1Ohv4hw_F75Gys1zjy_NPAC3dli1oJ.1_1 2019-12-09T09:52:00.105 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.098+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.30__multipart_key1.2~Y1Ohv4hw_F75Gys1zjy_NPAC3dli1oJ.2 2019-12-09T09:52:00.272 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.270+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.30__shadow_key1.2~Y1Ohv4hw_F75Gys1zjy_NPAC3dli1oJ.2_1 2019-12-09T09:52:00.362 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.360+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.30__multipart_key1.2~Y1Ohv4hw_F75Gys1zjy_NPAC3dli1oJ.3 2019-12-09T09:52:00.362 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.360+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.30__shadow_key1.2~Y1Ohv4hw_F75Gys1zjy_NPAC3dli1oJ.3_1 2019-12-09T09:52:00.362 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.360+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.30__multipart_key1.2~Y1Ohv4hw_F75Gys1zjy_NPAC3dli1oJ.4 2019-12-09T09:52:00.362 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.360+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.30__shadow_key1.2~Y1Ohv4hw_F75Gys1zjy_NPAC3dli1oJ.4_1 2019-12-09T09:52:00.363 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.360+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.30__multipart_key1.2~Y1Ohv4hw_F75Gys1zjy_NPAC3dli1oJ.5 2019-12-09T09:52:00.554 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.552+0000 7ff338754ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:52:00.650 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.648+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:52:00.697 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.695+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:00.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.711+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:00.722 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.720+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:00.775 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.773+0000 7ff338754ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:52:00.861 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.859+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:00.889 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.887+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:00.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:00.906+0000 7ff338754ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:01.005 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.003+0000 7ff338754ec0 20 remove_watcher() i=0 2019-12-09T09:52:01.005 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.003+0000 7ff338754ec0 2 removed watcher, disabling cache 2019-12-09T09:52:01.044 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.042+0000 7ff338754ec0 20 remove_watcher() i=1 2019-12-09T09:52:01.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.073+0000 7ff338754ec0 20 remove_watcher() i=2 2019-12-09T09:52:01.166 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.164+0000 7ff338754ec0 20 remove_watcher() i=3 2019-12-09T09:52:01.236 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.234+0000 7ff338754ec0 20 remove_watcher() i=4 2019-12-09T09:52:01.277 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.275+0000 7ff338754ec0 20 remove_watcher() i=5 2019-12-09T09:52:01.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.320+0000 7ff338754ec0 20 remove_watcher() i=6 2019-12-09T09:52:01.371 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.368+0000 7ff338754ec0 20 remove_watcher() i=7 2019-12-09T09:52:01.382 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T09:52:01.528 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.523+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.555 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.553+0000 7f1fc9fdeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:01.555 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.553+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.565 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.563+0000 7f1fc9fdeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:01.566 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.563+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.571 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.569+0000 7f1fc9fdeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:01.571 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.569+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.580 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.578+0000 7f1fc9fdeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:01.799 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.797+0000 7f1fc9fdeec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:52:01.799 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.797+0000 7f1fc9fdeec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:52:01.800 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.797+0000 7f1fc9fdeec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:52:01.800 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.797+0000 7f1fc9fdeec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:52:01.800 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.797+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.812 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.810+0000 7f1fc9fdeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:01.812 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.810+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.831+0000 7f1fc9fdeec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:52:01.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.831+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.846 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.844+0000 7f1fc9fdeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:01.847 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.844+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.862 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.860+0000 7f1fc9fdeec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:52:01.863 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.860+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.869 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.867+0000 7f1fc9fdeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:01.869 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.867+0000 7f1fc9fdeec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:52:01.870 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.867+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.875 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.872+0000 7f1fc9fdeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:01.875 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.872+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.884 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.882+0000 7f1fc9fdeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:01.885 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.882+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.902 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.900+0000 7f1fc9fdeec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:52:01.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.900+0000 7f1fc9fdeec0 20 zonegroup default 2019-12-09T09:52:01.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.900+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.906+0000 7f1fc9fdeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:01.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.906+0000 7f1fc9fdeec0 10 Cannot find current period zone using local zone 2019-12-09T09:52:01.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.906+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.917 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.915+0000 7f1fc9fdeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:01.917 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.915+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.924+0000 7f1fc9fdeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:01.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.924+0000 7f1fc9fdeec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:01.947 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.945+0000 7f1fc9fdeec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:52:01.948 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.945+0000 7f1fc9fdeec0 20 zone default found 2019-12-09T09:52:01.948 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.945+0000 7f1fc9fdeec0 20 started sync module instance, tier type = 2019-12-09T09:52:01.948 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:01.945+0000 7f1fc9fdeec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:52:02.343 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.340+0000 7f1fc9fdeec0 20 add_watcher() i=0 2019-12-09T09:52:02.343 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.340+0000 7f1fc9fdeec0 20 add_watcher() i=1 2019-12-09T09:52:02.343 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.340+0000 7f1fc9fdeec0 20 add_watcher() i=2 2019-12-09T09:52:02.343 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.340+0000 7f1fc9fdeec0 20 add_watcher() i=3 2019-12-09T09:52:02.343 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.340+0000 7f1fc9fdeec0 20 add_watcher() i=4 2019-12-09T09:52:02.343 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.340+0000 7f1fc9fdeec0 20 add_watcher() i=5 2019-12-09T09:52:02.344 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.340+0000 7f1fc9fdeec0 20 add_watcher() i=6 2019-12-09T09:52:02.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.376+0000 7f1fc9fdeec0 20 add_watcher() i=7 2019-12-09T09:52:02.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.376+0000 7f1fc9fdeec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:52:02.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.376+0000 7f1f87fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:52:02.389 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.387+0000 7f1faffff700 0 -- 172.21.15.35:0/3452951675 >> v2:172.21.15.35:3300/0 conn(0x5628cd3e5f20 msgr2=0x5628cd3e8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T09:52:02.591 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.589+0000 7f1faffff700 0 --2- 172.21.15.35:0/3452951675 >> v2:172.21.15.35:3300/0 conn(0x5628cd3e5f20 0x5628cd3e8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T09:52:02.649 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:02.646+0000 7f1fb7a09700 0 -- 172.21.15.35:0/4010236918 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x5628cd3ae0f0 msgr2=0x5628cd3ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T09:52:04.186 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.183+0000 7f1ef67fc700 20 reqs_thread_entry: start 2019-12-09T09:52:04.186 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.183+0000 7f1eeffff700 20 reqs_thread_entry: start 2019-12-09T09:52:04.186 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.184+0000 7f1fc9fdeec0 20 init_complete bucket index max shards: 0 2019-12-09T09:52:04.186 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.184+0000 7f1eee7fc700 20 reqs_thread_entry: start 2019-12-09T09:52:04.187 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.184+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:52:04.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.250+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:04.273 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.271+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:04.283 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.280+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:04.327 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.324+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:52:04.408 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.405+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:04.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.441+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:04.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.456+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:04.517 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.513+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:52:04.584 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.581+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:04.603 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.599+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:04.612 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.609+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:04.660 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.657+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:52:04.817 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.814+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:04.848 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.845+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:04.871 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.868+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:04.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:04.937+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:52:05.006 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.002+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:05.031 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.028+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:05.064 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.060+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:05.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.099+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:52:05.142 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.138+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:05.165 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.156+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:05.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.165+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:05.198 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.195+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:52:05.287 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.283+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:05.326 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.323+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:05.353 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.350+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:05.417 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.414+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:52:05.513 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.510+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:05.550 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.547+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:05.566 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.563+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:05.642 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.639+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:52:05.755 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.752+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:05.815 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.812+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:05.840 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.836+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:05.913 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.910+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:52:05.964 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.960+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:05.981 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.977+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:05.989 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:05.986+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.030 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.026+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:52:06.101 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.098+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.141 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.138+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:06.155 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.151+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.195 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.192+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:52:06.248 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.244+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.267 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.264+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:06.276 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.273+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.312 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.308+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:52:06.363 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.360+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.380 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.377+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:06.389 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.386+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.417+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:52:06.479 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.476+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.503 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.500+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:06.514 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.510+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.561+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:52:06.626 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.623+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.643 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.640+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:06.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.649+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.712 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.709+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:52:06.755 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.752+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.771 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.768+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:06.779 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.777+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.806 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.803+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:52:06.886 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.883+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.916 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.913+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:06.931 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.928+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:06.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:06.992+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:52:07.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.066+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.096+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:07.109 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.105+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.165 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.161+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:52:07.232 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.229+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.260 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.257+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:07.278 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.275+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.330 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.327+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:52:07.390 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.387+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.412 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.409+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:07.424 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.421+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.453+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:52:07.500 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.497+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.519 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.516+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:07.532 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.529+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.565 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.561+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:52:07.608 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.605+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.625 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.622+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:07.634 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.631+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.671 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.668+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:52:07.764 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.761+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.794 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.791+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:07.811 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.808+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.870 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.867+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:52:07.957 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.953+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:07.990 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:07.988+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:08.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.008+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.054+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:52:08.136 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.134+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.172+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:08.195 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.193+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.301 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.299+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:52:08.354 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.352+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.381 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.378+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:08.396 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.393+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.431+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:52:08.487 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.485+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.508 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.506+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:08.524 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.522+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.583 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.581+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:52:08.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.629+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.648 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.645+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:08.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.654+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.705 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.702+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:52:08.772 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.770+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.788 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.786+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:08.798 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.796+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.843 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.841+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:52:08.916 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.914+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.940 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.937+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:08.948 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.945+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:08.984 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:08.982+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:52:09.023 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.021+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:09.042 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.040+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:09.052 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.050+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:09.082 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.080+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:52:09.123 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.121+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:09.138 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.136+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:09.149 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.147+0000 7f1fc9fdeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:09.229 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.227+0000 7f1fc9fdeec0 20 remove_watcher() i=0 2019-12-09T09:52:09.229 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.227+0000 7f1fc9fdeec0 2 removed watcher, disabling cache 2019-12-09T09:52:09.308 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.306+0000 7f1fc9fdeec0 20 remove_watcher() i=1 2019-12-09T09:52:09.350 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.348+0000 7f1fc9fdeec0 20 remove_watcher() i=2 2019-12-09T09:52:09.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.419+0000 7f1fc9fdeec0 20 remove_watcher() i=3 2019-12-09T09:52:09.463 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.461+0000 7f1fc9fdeec0 20 remove_watcher() i=4 2019-12-09T09:52:09.505 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.503+0000 7f1fc9fdeec0 20 remove_watcher() i=5 2019-12-09T09:52:09.561 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.559+0000 7f1fc9fdeec0 20 remove_watcher() i=6 2019-12-09T09:52:09.657 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.655+0000 7f1fc9fdeec0 20 remove_watcher() i=7 2019-12-09T09:52:09.668 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T09:52:09.821 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.819+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:09.847 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.845+0000 7f45f005cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:09.847 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.845+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:09.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.855+0000 7f45f005cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:09.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.855+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:09.862 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.860+0000 7f45f005cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:09.862 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.860+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:09.872 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:09.871+0000 7f45f005cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:10.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.069+0000 7f45f005cec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:52:10.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.069+0000 7f45f005cec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:52:10.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.069+0000 7f45f005cec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:52:10.072 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.069+0000 7f45f005cec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:52:10.072 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.069+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.082 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.080+0000 7f45f005cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:10.082 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.080+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.103 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.100+0000 7f45f005cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:52:10.103 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.100+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.114+0000 7f45f005cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:10.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.114+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.159 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.157+0000 7f45f005cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:52:10.159 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.157+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.166+0000 7f45f005cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:10.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.166+0000 7f45f005cec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:52:10.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.166+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.176 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.174+0000 7f45f005cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:10.176 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.174+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.193 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.187+0000 7f45f005cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:10.193 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.187+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.207 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.205+0000 7f45f005cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:52:10.208 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.205+0000 7f45f005cec0 20 zonegroup default 2019-12-09T09:52:10.208 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.205+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.215 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.213+0000 7f45f005cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:10.216 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.213+0000 7f45f005cec0 10 Cannot find current period zone using local zone 2019-12-09T09:52:10.216 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.213+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.222+0000 7f45f005cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:10.225 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.222+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.234 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.231+0000 7f45f005cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:10.234 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.231+0000 7f45f005cec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:10.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.249+0000 7f45f005cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:52:10.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.249+0000 7f45f005cec0 20 zone default found 2019-12-09T09:52:10.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.249+0000 7f45f005cec0 20 started sync module instance, tier type = 2019-12-09T09:52:10.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.249+0000 7f45f005cec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:52:10.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.650+0000 7f45f005cec0 20 add_watcher() i=0 2019-12-09T09:52:10.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.650+0000 7f45f005cec0 20 add_watcher() i=1 2019-12-09T09:52:10.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.650+0000 7f45f005cec0 20 add_watcher() i=2 2019-12-09T09:52:10.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.650+0000 7f45f005cec0 20 add_watcher() i=3 2019-12-09T09:52:10.653 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.650+0000 7f45f005cec0 20 add_watcher() i=4 2019-12-09T09:52:10.653 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.650+0000 7f45f005cec0 20 add_watcher() i=5 2019-12-09T09:52:10.653 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.650+0000 7f45f005cec0 20 add_watcher() i=6 2019-12-09T09:52:10.739 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.737+0000 7f45dd286700 0 -- 172.21.15.35:0/3975680935 >> [v2:172.21.15.22:6802/9813,v1:172.21.15.22:6803/9813] conn(0x7f45c0030af0 msgr2=0x7f45c0033020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:52:10.816 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.813+0000 7f45f005cec0 20 add_watcher() i=7 2019-12-09T09:52:10.816 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.813+0000 7f45f005cec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:52:10.816 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.814+0000 7f45adfdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:52:10.836 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:10.834+0000 7f45dda87700 0 -- 172.21.15.35:0/2618105562 >> v1:172.21.15.22:6789/0 conn(0x557db7ce09d0 legacy=0x557db7cdfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:52:12.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.381+0000 7f451ffff700 20 reqs_thread_entry: start 2019-12-09T09:52:12.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.381+0000 7f451dffb700 20 reqs_thread_entry: start 2019-12-09T09:52:12.384 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.381+0000 7f45f005cec0 20 init_complete bucket index max shards: 0 2019-12-09T09:52:12.384 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.381+0000 7f450bfff700 20 reqs_thread_entry: start 2019-12-09T09:52:12.384 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.381+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:52:12.436 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.434+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:12.453 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.451+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:12.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.459+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:12.504 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.502+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:52:12.575 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.573+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:12.598 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.596+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:12.613 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.608+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:12.653 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.650+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:52:12.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.706+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:12.731 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.729+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:12.742 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.740+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:12.793 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.791+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:52:12.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.849+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:12.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.885+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:12.907 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.905+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:12.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:12.942+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:52:13.022 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.020+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:13.051 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.049+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:13.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.069+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:13.131 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.130+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:52:13.223 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.219+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:13.257 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.255+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:13.278 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.276+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:13.343 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.341+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:52:13.410 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.408+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:13.434 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.432+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:13.445 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.440+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:13.494 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.492+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:52:13.565 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.563+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:13.593 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.590+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:13.606 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.604+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:13.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.649+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:52:13.757 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.754+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:13.788 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.786+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:13.804 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.802+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:13.859 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.857+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:52:13.937 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.935+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:13.961 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.959+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:13.973 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:13.971+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.024 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.022+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:52:14.166 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.164+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.197 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.195+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:14.231 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.230+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.288 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.281+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:52:14.332 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.330+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.351 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.349+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:14.361 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.359+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.397 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.395+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:52:14.442 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.440+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.458+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:14.468 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.466+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.521 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.519+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:52:14.567 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.565+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.587 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.585+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:14.595 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.593+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.636 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.634+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:52:14.728 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.719+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.753 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.752+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:14.770 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.768+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.837 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.835+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:52:14.917 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.915+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:14.938 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.936+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:14.950 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:14.948+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.004 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.002+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:52:15.067 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.065+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.084+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:15.096 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.094+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.141 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.139+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:52:15.223 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.221+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.251 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.248+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:15.263 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.261+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.339 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.337+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:52:15.378 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.375+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.397 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.394+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:15.407 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.405+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.446 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.444+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:52:15.497 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.494+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.513 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.511+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:15.522 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.520+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.566 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.563+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:52:15.614 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.611+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.636 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.634+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:15.648 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.646+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.691+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:52:15.760 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.758+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.780 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.778+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:15.790 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.788+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.839+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:52:15.883 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.881+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.900 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.898+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:15.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.907+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.938 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.936+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:52:15.974 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.972+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:15.990 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.988+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:15.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:15.997+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.032 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.030+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:52:16.082 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.079+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.100 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.097+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:16.108 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.106+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.160 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.158+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:52:16.238 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.236+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.276 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.274+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:16.299 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.297+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.359 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.356+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:52:16.405 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.402+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.429 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.427+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:16.445 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.443+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.478 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.475+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:52:16.555 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.553+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.630 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.628+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:16.641 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.639+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.695 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.693+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:52:16.754 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.751+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.788 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.786+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:16.808 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.806+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.846 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.844+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:52:16.889 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.887+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.905 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.903+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:16.914 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.912+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:16.945 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:16.942+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:52:17.044 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.042+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:17.080 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.078+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:17.100 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.098+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:17.166 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.163+0000 7f45f005cec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:52:17.246 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.244+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:17.270 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.268+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:17.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.283+0000 7f45f005cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:17.381 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.378+0000 7f45f005cec0 20 remove_watcher() i=0 2019-12-09T09:52:17.381 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.378+0000 7f45f005cec0 2 removed watcher, disabling cache 2019-12-09T09:52:17.451 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.449+0000 7f45f005cec0 20 remove_watcher() i=1 2019-12-09T09:52:17.513 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.511+0000 7f45f005cec0 20 remove_watcher() i=2 2019-12-09T09:52:17.591 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.589+0000 7f45f005cec0 20 remove_watcher() i=3 2019-12-09T09:52:17.655 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.653+0000 7f45f005cec0 20 remove_watcher() i=4 2019-12-09T09:52:17.712 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.710+0000 7f45f005cec0 20 remove_watcher() i=5 2019-12-09T09:52:17.770 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.768+0000 7f45f005cec0 20 remove_watcher() i=6 2019-12-09T09:52:17.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.832+0000 7f45f005cec0 20 remove_watcher() i=7 2019-12-09T09:52:17.842 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T09:52:17.958 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.953+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:17.979 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.976+0000 7f9a73c91ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:17.979 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.976+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:17.986 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.984+0000 7f9a73c91ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:17.986 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.984+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:17.992 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.990+0000 7f9a73c91ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:17.993 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.990+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:17.998+0000 7f9a73c91ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:18.209 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.207+0000 7f9a73c91ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:52:18.209 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.207+0000 7f9a73c91ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:52:18.209 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.207+0000 7f9a73c91ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:52:18.209 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.207+0000 7f9a73c91ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:52:18.210 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.207+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.222 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.220+0000 7f9a73c91ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:18.222 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.220+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.240 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.238+0000 7f9a73c91ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:52:18.241 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.238+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.254 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.252+0000 7f9a73c91ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:18.254 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.252+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.272 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.270+0000 7f9a73c91ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:52:18.272 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.270+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.282 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.280+0000 7f9a73c91ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:18.283 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.280+0000 7f9a73c91ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:52:18.283 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.280+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.287 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.285+0000 7f9a73c91ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:18.288 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.285+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.295+0000 7f9a73c91ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:18.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.295+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.314+0000 7f9a73c91ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:52:18.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.314+0000 7f9a73c91ec0 20 zonegroup default 2019-12-09T09:52:18.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.314+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.321 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.319+0000 7f9a73c91ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:18.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.319+0000 7f9a73c91ec0 10 Cannot find current period zone using local zone 2019-12-09T09:52:18.322 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.319+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.327 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.325+0000 7f9a73c91ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:52:18.327 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.325+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.341 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.339+0000 7f9a73c91ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:52:18.341 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.339+0000 7f9a73c91ec0 20 rados->read ofs=0 len=0 2019-12-09T09:52:18.361 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.358+0000 7f9a73c91ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:52:18.361 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.358+0000 7f9a73c91ec0 20 zone default found 2019-12-09T09:52:18.361 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.358+0000 7f9a73c91ec0 20 started sync module instance, tier type = 2019-12-09T09:52:18.362 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.358+0000 7f9a73c91ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:52:18.722 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.716+0000 7f9a73c91ec0 20 add_watcher() i=0 2019-12-09T09:52:18.722 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.716+0000 7f9a73c91ec0 20 add_watcher() i=1 2019-12-09T09:52:18.722 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.716+0000 7f9a73c91ec0 20 add_watcher() i=2 2019-12-09T09:52:18.723 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.716+0000 7f9a73c91ec0 20 add_watcher() i=3 2019-12-09T09:52:18.723 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.716+0000 7f9a73c91ec0 20 add_watcher() i=4 2019-12-09T09:52:18.723 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.716+0000 7f9a73c91ec0 20 add_watcher() i=5 2019-12-09T09:52:18.723 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.721+0000 7f9a73c91ec0 20 add_watcher() i=6 2019-12-09T09:52:18.760 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.758+0000 7f9a73c91ec0 20 add_watcher() i=7 2019-12-09T09:52:18.760 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.758+0000 7f9a73c91ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:52:18.761 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.758+0000 7f9a317e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:52:18.764 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.762+0000 7f9a60ebb700 0 -- >> v2:172.21.15.35:3300/0 conn(0x564ceb579820 msgr2=0x564ceb57bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T09:52:18.825 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:18.823+0000 7f9a616bc700 0 -- 172.21.15.35:0/873764522 >> [v2:172.21.15.22:6800/8484,v1:172.21.15.22:6801/8484] conn(0x7f99a4022470 msgr2=0x7f99a4024960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:52:20.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.441+0000 7f999b7fe700 20 reqs_thread_entry: start 2019-12-09T09:52:20.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.441+0000 7f99997fa700 20 reqs_thread_entry: start 2019-12-09T09:52:20.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.441+0000 7f9a73c91ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:52:20.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.441+0000 7f998b7fe700 20 reqs_thread_entry: start 2019-12-09T09:52:20.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.442+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:52:20.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.487+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:20.511 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.509+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:20.530 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.528+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:20.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.572+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:52:20.615 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.613+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:20.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.628+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:20.640 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.638+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:20.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.682+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:52:20.751 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.749+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:20.771 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.769+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:20.779 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.777+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:20.842 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.840+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:52:20.913 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.910+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:20.940 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.938+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:20.952 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:20.950+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:21.006 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.004+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:52:21.078 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.076+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:21.107 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.101+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:21.113 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.111+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:21.158 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.156+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:52:21.243 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.241+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:21.280 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.277+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:21.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.295+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:21.363 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.361+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:52:21.435 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.433+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:21.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.459+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:21.476 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.474+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:21.534 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.532+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:52:21.590 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.588+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:21.613 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.611+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:21.627 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.624+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:21.666 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.664+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:52:21.767 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.765+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:21.819 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.817+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:21.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.849+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:21.968 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:21.963+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:52:22.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.008+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.035 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.033+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:22.044 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.042+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.085 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.083+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:52:22.157 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.155+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.190 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.188+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:22.200 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.198+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.243 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.241+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:52:22.295 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.289+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.309 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.307+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:22.321 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.320+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.360 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.358+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:52:22.399 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.397+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.415 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.413+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:22.424 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.421+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.458 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.456+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:52:22.506 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.504+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.529 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.527+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:22.537 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.536+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.576 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.575+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:52:22.640 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.638+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.662 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.660+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:22.673 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.671+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.725 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.723+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:52:22.793 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.791+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.816 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.813+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:22.827 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.825+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:22.883 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.881+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:52:22.987 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:22.985+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.020 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.018+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:23.040 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.038+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.113 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.111+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:52:23.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.166+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.194 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.192+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:23.209 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.207+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.238 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.236+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:52:23.330 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.328+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.369 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.367+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:23.405 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.387+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.455 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.453+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:52:23.532 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.530+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.558 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.556+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:23.576 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.574+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.621+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:52:23.683 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.681+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.705 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.703+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:23.716 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.714+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.765 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.763+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:52:23.804 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.802+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.820 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.818+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:23.828 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.826+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.878 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.875+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:52:23.931 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.929+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:23.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.953+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:23.966 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:23.964+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.028 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.026+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:52:24.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.084+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.109 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.103+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:24.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.115+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.150 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.148+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:52:24.241 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.239+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.279 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.277+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:24.298 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.296+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.353 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.351+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:52:24.414 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.412+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.436 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.434+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:24.449 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.447+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.513 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.509+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:52:24.554 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.552+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.572 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.570+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:24.581 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.579+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.616 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.614+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:52:24.682 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.680+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.711 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.708+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:24.721 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.719+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.756 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.754+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:52:24.798 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.796+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.821 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.818+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:24.833 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.831+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.870 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.868+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:52:24.958 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.956+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:24.976 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.973+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:24.985 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:24.982+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:25.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.096+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:52:25.160 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.158+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:25.179 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.177+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:25.194 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.191+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:25.248 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.246+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:52:25.337 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.332+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:25.350 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.348+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:52:25.358 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.356+0000 7f9a73c91ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:52:25.470 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.468+0000 7f9a73c91ec0 20 remove_watcher() i=0 2019-12-09T09:52:25.471 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.468+0000 7f9a73c91ec0 2 removed watcher, disabling cache 2019-12-09T09:52:25.543 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.540+0000 7f9a73c91ec0 20 remove_watcher() i=1 2019-12-09T09:52:25.601 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.598+0000 7f9a73c91ec0 20 remove_watcher() i=2 2019-12-09T09:52:25.675 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.672+0000 7f9a73c91ec0 20 remove_watcher() i=3 2019-12-09T09:52:25.745 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.743+0000 7f9a73c91ec0 20 remove_watcher() i=4 2019-12-09T09:52:25.799 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.797+0000 7f9a73c91ec0 20 remove_watcher() i=5 2019-12-09T09:52:25.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.855+0000 7f9a73c91ec0 20 remove_watcher() i=6 2019-12-09T09:52:25.914 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:52:25.912+0000 7f9a73c91ec0 20 remove_watcher() i=7 2019-12-09T09:52:25.923 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2019-12-09T09:52:26.854 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :clean 2019-12-09T09:52:27.451 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :compileJava 2019-12-09T09:52:27.452 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :processResources 2019-12-09T09:52:27.452 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :classes 2019-12-09T09:52:27.853 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:52:27.854 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :compileTestJava 2019-12-09T09:52:27.854 INFO:teuthology.orchestra.run.smithi035.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T09:52:27.854 INFO:teuthology.orchestra.run.smithi035.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T09:52:27.854 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:52:27.854 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T09:52:27.854 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :testClasses 2019-12-09T09:52:30.551 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:52:30.552 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :test 2019-12-09T09:52:30.552 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:52:30.552 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2019-12-09T09:52:33.051 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:52:33.051 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2019-12-09T09:52:39.451 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:52:39.451 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2019-12-09T09:52:42.051 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:52:42.051 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2019-12-09T09:52:48.651 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:52:48.651 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2019-12-09T09:52:50.751 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:52:50.751 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2019-12-09T09:52:57.451 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:52:57.451 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2019-12-09T09:53:05.551 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:05.551 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2019-12-09T09:53:07.051 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:07.051 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2019-12-09T09:53:12.451 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:12.452 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2019-12-09T09:53:19.051 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:19.051 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2019-12-09T09:53:19.351 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:19.352 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2019-12-09T09:53:21.751 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:21.751 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2019-12-09T09:53:27.851 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:27.851 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2019-12-09T09:53:41.051 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:41.051 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2019-12-09T09:53:42.851 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:42.852 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2019-12-09T09:53:47.251 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:47.251 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2019-12-09T09:53:47.551 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:47.552 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2019-12-09T09:53:48.411 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:48.411 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------- 2019-12-09T09:53:48.411 INFO:teuthology.orchestra.run.smithi035.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2019-12-09T09:53:48.412 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------- 2019-12-09T09:53:48.412 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:53:48.412 INFO:teuthology.orchestra.run.smithi035.stdout:BUILD SUCCESSFUL in 1m 22s 2019-12-09T09:53:48.413 INFO:teuthology.orchestra.run.smithi035.stdout:5 actionable tasks: 5 executed 2019-12-09T09:53:48.420 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T09:53:48.655 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.653+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:48.700 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.698+0000 7fc60ba19ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:48.700 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.698+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:48.711 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.709+0000 7fc60ba19ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:48.711 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.709+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:48.722 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.716+0000 7fc60ba19ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:48.722 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.716+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:48.731 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.729+0000 7fc60ba19ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:53:48.939 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.937+0000 7fc60ba19ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:53:48.939 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.937+0000 7fc60ba19ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:53:48.939 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.937+0000 7fc60ba19ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:53:48.939 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.937+0000 7fc60ba19ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:53:48.940 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.937+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:48.952 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.949+0000 7fc60ba19ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:53:48.952 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.950+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:48.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.963+0000 7fc60ba19ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:53:48.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.963+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:48.975 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.973+0000 7fc60ba19ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:53:48.976 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.973+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:48.994 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.992+0000 7fc60ba19ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:53:48.994 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.992+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:49.003 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.998+0000 7fc60ba19ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:49.003 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.998+0000 7fc60ba19ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:53:49.003 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:48.998+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:49.004 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.002+0000 7fc60ba19ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:49.004 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.002+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:49.015 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.013+0000 7fc60ba19ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:53:49.015 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.013+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:49.035 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.033+0000 7fc60ba19ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:53:49.036 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.033+0000 7fc60ba19ec0 20 zonegroup default 2019-12-09T09:53:49.036 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.033+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:49.045 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.043+0000 7fc60ba19ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:49.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.043+0000 7fc60ba19ec0 10 Cannot find current period zone using local zone 2019-12-09T09:53:49.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.043+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:49.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.054+0000 7fc60ba19ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:49.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.054+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:49.073 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.071+0000 7fc60ba19ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:53:49.074 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.071+0000 7fc60ba19ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:49.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.090+0000 7fc60ba19ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:53:49.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.090+0000 7fc60ba19ec0 20 zone default found 2019-12-09T09:53:49.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.090+0000 7fc60ba19ec0 20 started sync module instance, tier type = 2019-12-09T09:53:49.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.090+0000 7fc60ba19ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:53:49.473 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.470+0000 7fc60ba19ec0 20 add_watcher() i=0 2019-12-09T09:53:49.473 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.470+0000 7fc60ba19ec0 20 add_watcher() i=1 2019-12-09T09:53:49.473 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.470+0000 7fc60ba19ec0 20 add_watcher() i=2 2019-12-09T09:53:49.473 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.470+0000 7fc60ba19ec0 20 add_watcher() i=3 2019-12-09T09:53:49.474 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.470+0000 7fc60ba19ec0 20 add_watcher() i=4 2019-12-09T09:53:49.474 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.470+0000 7fc60ba19ec0 20 add_watcher() i=5 2019-12-09T09:53:49.474 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.470+0000 7fc60ba19ec0 20 add_watcher() i=6 2019-12-09T09:53:49.523 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.521+0000 7fc60ba19ec0 20 add_watcher() i=7 2019-12-09T09:53:49.523 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.521+0000 7fc60ba19ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:53:49.523 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.521+0000 7fc5c8fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:53:49.544 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.542+0000 7fc5f8c43700 0 -- 172.21.15.35:0/4110356063 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x55edf45e5560 msgr2=0x55edf4664130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:53:49.654 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:49.652+0000 7fc5f9444700 0 -- 172.21.15.35:0/4110356063 >> [v2:172.21.15.35:6816/9792,v1:172.21.15.35:6817/9792] conn(0x55edf4670a20 msgr2=0x55edf4672e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:53:51.225 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.220+0000 7fc53affd700 20 reqs_thread_entry: start 2019-12-09T09:53:51.226 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.221+0000 7fc538ff9700 20 reqs_thread_entry: start 2019-12-09T09:53:51.226 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.221+0000 7fc60ba19ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:53:51.226 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.222+0000 7fc52effd700 20 reqs_thread_entry: start 2019-12-09T09:53:51.226 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.222+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:53:51.301 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.298+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:51.321 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.319+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:51.341 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.332+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:51.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.382+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:53:51.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.464+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:51.501 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.499+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:51.529 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.527+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:51.583 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.581+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:53:51.657 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.655+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:51.685 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.683+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:51.698 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.696+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:51.756 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.754+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:53:51.850 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.848+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:51.881 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.879+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:51.902 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.900+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:51.975 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:51.973+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:53:52.058 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.056+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.085 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.083+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:52.098 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.096+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.150 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.148+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:53:52.205 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.203+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.222+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:52.236 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.234+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.279 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.277+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:53:52.352 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.350+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.369 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.367+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:52.377 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.375+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.406 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.404+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:53:52.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.475+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.498 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.496+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:52.512 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.510+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.543 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.541+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:53:52.589 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.587+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.608+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:52.620 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.618+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.650+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:53:52.720 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.718+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.756 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.754+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:52.768 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.766+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.815 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.813+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:53:52.894 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.892+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.914 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.912+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:52.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.925+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:52.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:52.963+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:53:53.024 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.022+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.055 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.053+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:53.070 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.068+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.135 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.133+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:53:53.213 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.211+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.237 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.235+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:53.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.250+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.283+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:53:53.328 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.326+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.348 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.346+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:53.372 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.371+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.410 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.408+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:53:53.461 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.459+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.475+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:53.486 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.484+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.536 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.534+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:53:53.595 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.593+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.615 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.613+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:53.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.621+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.689 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.687+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:53:53.740 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.738+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.761 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.759+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:53.770 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.768+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.809 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.807+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:53:53.869 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.867+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.887 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.885+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:53.897 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.895+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.935 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.933+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:53:53.973 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.971+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:53.990 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.988+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:54.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:53.997+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.027+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:53:54.082 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.080+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.097+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:54.108 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.106+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.151 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.149+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:53:54.203 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.201+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.225 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.223+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:54.236 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.234+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.278 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.276+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:53:54.345 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.343+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.373 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.371+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:54.389 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.387+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.418+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:53:54.478 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.476+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.504 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.502+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:54.518 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.516+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.566 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.564+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:53:54.636 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.634+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.659 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.657+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:54.671 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.669+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.710 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.708+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:53:54.749 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.747+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.766 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.764+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:54.779 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.777+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.810 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.808+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:53:54.900 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.898+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.931 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.929+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:54.945 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.942+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:54.998 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:54.996+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:53:55.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.069+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:55.107 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.105+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:55.126 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.124+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:55.190 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.188+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:53:55.279 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.277+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:55.314 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.312+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:55.333 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.331+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:55.394 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.392+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:53:55.480 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.478+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:55.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.507+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:55.522 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.520+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:55.577 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.575+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:53:55.670 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.667+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:55.703 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.701+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:55.713 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.711+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:55.749 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.747+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:53:55.814 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.811+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:55.835 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.833+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:55.843 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.841+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:55.887 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.885+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:53:55.998 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:55.996+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:56.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.045+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:56.065 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.063+0000 7fc60ba19ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:56.168 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.166+0000 7fc60ba19ec0 20 remove_watcher() i=0 2019-12-09T09:53:56.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.166+0000 7fc60ba19ec0 2 removed watcher, disabling cache 2019-12-09T09:53:56.210 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.208+0000 7fc60ba19ec0 20 remove_watcher() i=1 2019-12-09T09:53:56.257 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.255+0000 7fc60ba19ec0 20 remove_watcher() i=2 2019-12-09T09:53:56.313 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.311+0000 7fc60ba19ec0 20 remove_watcher() i=3 2019-12-09T09:53:56.375 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.373+0000 7fc60ba19ec0 20 remove_watcher() i=4 2019-12-09T09:53:56.421 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.419+0000 7fc60ba19ec0 20 remove_watcher() i=5 2019-12-09T09:53:56.482 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.480+0000 7fc60ba19ec0 20 remove_watcher() i=6 2019-12-09T09:53:56.556 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.553+0000 7fc60ba19ec0 20 remove_watcher() i=7 2019-12-09T09:53:56.568 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T09:53:56.740 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.738+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:56.774 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.772+0000 7fe311678ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:56.775 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.772+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:56.786 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.784+0000 7fe311678ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:56.786 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.784+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:56.791 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.789+0000 7fe311678ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:56.791 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.789+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:56.800 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:56.798+0000 7fe311678ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:53:57.038 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.036+0000 7fe311678ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:53:57.038 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.036+0000 7fe311678ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:53:57.038 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.036+0000 7fe311678ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:53:57.038 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.036+0000 7fe311678ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:53:57.038 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.036+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.053 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.051+0000 7fe311678ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:53:57.053 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.051+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.073 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.071+0000 7fe311678ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:53:57.074 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.071+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.089 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.087+0000 7fe311678ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:53:57.089 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.087+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.109 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.107+0000 7fe311678ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:53:57.109 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.107+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.119 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.118+0000 7fe311678ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:57.120 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.118+0000 7fe311678ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:53:57.120 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.118+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.133 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.127+0000 7fe311678ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:57.133 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.127+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.144+0000 7fe311678ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:53:57.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.144+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.162 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.160+0000 7fe311678ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:53:57.162 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.160+0000 7fe311678ec0 20 zonegroup default 2019-12-09T09:53:57.162 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.160+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.170 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.168+0000 7fe311678ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:57.171 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.168+0000 7fe311678ec0 10 Cannot find current period zone using local zone 2019-12-09T09:53:57.171 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.168+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.179 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.177+0000 7fe311678ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:53:57.179 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.177+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.190 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.188+0000 7fe311678ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:53:57.190 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.188+0000 7fe311678ec0 20 rados->read ofs=0 len=0 2019-12-09T09:53:57.211 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.209+0000 7fe311678ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:53:57.211 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.209+0000 7fe311678ec0 20 zone default found 2019-12-09T09:53:57.211 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.209+0000 7fe311678ec0 20 started sync module instance, tier type = 2019-12-09T09:53:57.211 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.209+0000 7fe311678ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:53:57.599 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.597+0000 7fe311678ec0 20 add_watcher() i=0 2019-12-09T09:53:57.599 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.597+0000 7fe311678ec0 20 add_watcher() i=1 2019-12-09T09:53:57.599 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.597+0000 7fe311678ec0 20 add_watcher() i=2 2019-12-09T09:53:57.599 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.597+0000 7fe311678ec0 20 add_watcher() i=3 2019-12-09T09:53:57.600 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.597+0000 7fe311678ec0 20 add_watcher() i=4 2019-12-09T09:53:57.600 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.597+0000 7fe311678ec0 20 add_watcher() i=5 2019-12-09T09:53:57.600 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.597+0000 7fe311678ec0 20 add_watcher() i=6 2019-12-09T09:53:57.638 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.636+0000 7fe311678ec0 20 add_watcher() i=7 2019-12-09T09:53:57.638 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.636+0000 7fe311678ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:53:57.639 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.636+0000 7fe2cf7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:53:57.648 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.646+0000 7fe2ff0a3700 0 -- 172.21.15.35:0/3670285505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2216cf6d0 msgr2=0x55d2216f17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:53:57.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:57.679+0000 7fe2ff0a3700 0 -- 172.21.15.35:0/518889377 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x55d2216f4350 msgr2=0x55d2216f2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:53:59.130 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.128+0000 7fe23dffb700 20 reqs_thread_entry: start 2019-12-09T09:53:59.130 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.128+0000 7fe2377fe700 20 reqs_thread_entry: start 2019-12-09T09:53:59.131 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.128+0000 7fe311678ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:53:59.131 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.129+0000 7fe235ffb700 20 reqs_thread_entry: start 2019-12-09T09:53:59.131 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.129+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:53:59.218 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.216+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:59.254 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.252+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:59.276 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.274+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:59.337 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.335+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:53:59.406 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.404+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:59.437 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.435+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:59.447 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.445+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:59.479 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.477+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:53:59.544 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.541+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:59.566 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.564+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:59.579 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.577+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:59.620 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.618+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:53:59.692 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.690+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:59.719 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.716+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:59.732 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.730+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:59.792 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.790+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:53:59.863 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.860+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:59.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.886+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:53:59.895 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.893+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:53:59.951 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:53:59.949+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:54:00.036 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.034+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.061 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.059+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:00.078 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.076+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.112 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.110+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:54:00.157 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.155+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.173+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:00.184 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.182+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.216 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.214+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:54:00.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.279+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.309 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.307+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:00.324 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.321+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.376 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.374+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:54:00.428 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.425+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.447 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.445+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:00.459 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.457+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.497 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.495+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:54:00.546 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.544+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.564 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.562+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:00.573 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.571+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.612 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.610+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:54:00.650 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.648+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.667 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.665+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:00.676 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.674+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.711 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.708+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:54:00.773 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.771+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.803 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.801+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:00.814 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.812+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.848+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:54:00.898 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.891+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.915 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.913+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:00.926 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.924+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:00.962 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:00.960+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:54:01.037 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.035+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.058 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.056+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:01.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.069+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.112 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.110+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:54:01.164 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.162+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.184 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.182+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:01.194 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.192+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.234 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.232+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:54:01.286 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.284+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.304 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.301+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:01.317 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.314+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.364 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.362+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:54:01.437 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.435+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.464 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.462+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:01.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.475+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.524 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.521+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:54:01.637 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.635+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.670 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.668+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:01.683 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.681+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.712+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:54:01.805 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.803+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.822 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.820+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:01.831 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.829+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.881 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.879+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:54:01.940 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.938+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:01.967 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.965+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:01.980 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:01.978+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:02.042 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.040+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:54:02.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.109+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:02.155 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.153+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:02.168 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.166+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:02.348 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.346+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:54:02.415 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.413+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:02.437 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.435+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:02.447 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.445+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:02.487 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.485+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:54:02.536 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.534+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:02.554 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.552+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:02.563 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.561+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:02.594 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.592+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:54:02.659 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.657+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:02.677 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.675+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:02.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.682+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:02.732 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.730+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:54:02.819 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.816+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:02.845 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.842+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:02.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.856+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:02.905 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.903+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:54:02.973 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.970+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:02.993+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:03.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.008+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.045+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:54:03.125 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.123+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.159 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.157+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:03.178 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.176+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.226 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.224+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:54:03.286 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.284+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.308 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.305+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:03.321 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.318+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.355 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.353+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:54:03.399 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.397+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.419 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.417+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:03.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.429+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.464 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.462+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:54:03.550 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.548+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.589 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.587+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:03.613 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.611+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.682+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:54:03.784 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.782+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.816 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.814+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:03.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.832+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.886 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.884+0000 7fe311678ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:54:03.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.939+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:03.972 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.969+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:03.980 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:03.978+0000 7fe311678ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:04.114 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.112+0000 7fe311678ec0 20 remove_watcher() i=0 2019-12-09T09:54:04.114 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.112+0000 7fe311678ec0 2 removed watcher, disabling cache 2019-12-09T09:54:04.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.166+0000 7fe311678ec0 20 remove_watcher() i=1 2019-12-09T09:54:04.216 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.213+0000 7fe311678ec0 20 remove_watcher() i=2 2019-12-09T09:54:04.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.305+0000 7fe311678ec0 20 remove_watcher() i=3 2019-12-09T09:54:04.358 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.356+0000 7fe311678ec0 20 remove_watcher() i=4 2019-12-09T09:54:04.412 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.410+0000 7fe311678ec0 20 remove_watcher() i=5 2019-12-09T09:54:04.447 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.445+0000 7fe311678ec0 20 remove_watcher() i=6 2019-12-09T09:54:04.488 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.485+0000 7fe311678ec0 20 remove_watcher() i=7 2019-12-09T09:54:04.493 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T09:54:04.601 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.599+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.621+0000 7f799af96ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:04.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.621+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.629+0000 7f799af96ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:04.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.629+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.635 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.633+0000 7f799af96ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:04.636 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.633+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.643 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.641+0000 7f799af96ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:54:04.867 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.865+0000 7f799af96ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:54:04.868 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.865+0000 7f799af96ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:54:04.868 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.865+0000 7f799af96ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:54:04.868 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.865+0000 7f799af96ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:54:04.868 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.865+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.873+0000 7f799af96ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:54:04.880 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.873+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.896 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.894+0000 7f799af96ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:54:04.896 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.894+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.905+0000 7f799af96ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:54:04.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.905+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.928 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.926+0000 7f799af96ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:54:04.929 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.926+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.935 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.933+0000 7f799af96ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:04.935 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.933+0000 7f799af96ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:54:04.935 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.933+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.939+0000 7f799af96ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:04.941 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.939+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.947 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.945+0000 7f799af96ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:54:04.948 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.945+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.968 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.966+0000 7f799af96ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:54:04.968 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.966+0000 7f799af96ec0 20 zonegroup default 2019-12-09T09:54:04.968 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.966+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.977 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.975+0000 7f799af96ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:04.978 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.975+0000 7f799af96ec0 10 Cannot find current period zone using local zone 2019-12-09T09:54:04.978 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.975+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.984 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.981+0000 7f799af96ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:04.984 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.981+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:04.995 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.993+0000 7f799af96ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:54:04.995 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:04.993+0000 7f799af96ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:05.014 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.011+0000 7f799af96ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:54:05.014 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.011+0000 7f799af96ec0 20 zone default found 2019-12-09T09:54:05.014 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.011+0000 7f799af96ec0 20 started sync module instance, tier type = 2019-12-09T09:54:05.014 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.011+0000 7f799af96ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:54:05.489 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.487+0000 7f799af96ec0 20 add_watcher() i=0 2019-12-09T09:54:05.489 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.487+0000 7f799af96ec0 20 add_watcher() i=1 2019-12-09T09:54:05.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.487+0000 7f799af96ec0 20 add_watcher() i=2 2019-12-09T09:54:05.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.487+0000 7f799af96ec0 20 add_watcher() i=3 2019-12-09T09:54:05.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.487+0000 7f799af96ec0 20 add_watcher() i=4 2019-12-09T09:54:05.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.487+0000 7f799af96ec0 20 add_watcher() i=5 2019-12-09T09:54:05.523 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.520+0000 7f799af96ec0 20 add_watcher() i=6 2019-12-09T09:54:05.542 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.540+0000 7f799af96ec0 20 add_watcher() i=7 2019-12-09T09:54:05.542 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.540+0000 7f799af96ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:54:05.542 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.540+0000 7f7958fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:54:05.550 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.548+0000 7f79889c1700 0 -- 172.21.15.35:0/1585448309 >> v2:172.21.15.35:3300/0 conn(0x5636f2465530 msgr2=0x5636f24648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T09:54:05.619 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:05.617+0000 7f79881c0700 0 -- 172.21.15.35:0/2316489825 >> [v2:172.21.15.35:6808/9789,v1:172.21.15.35:6809/9789] conn(0x5636f2497b40 msgr2=0x5636f2499f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T09:54:07.511 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.508+0000 7f78caffd700 20 reqs_thread_entry: start 2019-12-09T09:54:07.511 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.509+0000 7f78c8ff9700 20 reqs_thread_entry: start 2019-12-09T09:54:07.511 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.509+0000 7f799af96ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:54:07.512 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.510+0000 7f78beffd700 20 reqs_thread_entry: start 2019-12-09T09:54:07.512 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.510+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:54:07.580 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.578+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:07.607 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.605+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:07.620 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.618+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:07.662 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.659+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:54:07.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.706+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:07.729 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.727+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:07.738 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.736+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:07.776 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.768+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:54:07.833 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.830+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:07.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.849+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:07.860 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.858+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:07.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.901+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:54:07.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:07.994+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.040 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.038+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:08.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.054+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.109+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:54:08.194 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.192+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.223 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.221+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:08.234 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.232+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.275 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.273+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:54:08.357 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.355+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.389 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.387+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:08.405 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.403+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.457 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.455+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:54:08.544 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.542+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.572+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:08.586 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.584+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.641 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.639+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:54:08.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.682+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.705 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.702+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:08.715 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.713+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.758 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.756+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:54:08.870 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.868+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.899 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.897+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:08.915 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.913+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:08.954 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:08.952+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:54:09.020 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.017+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.055+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:09.083 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.081+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.128 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.126+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:54:09.185 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.183+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.205 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.203+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:09.214 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.212+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.258 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.256+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:54:09.338 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.336+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.371 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.369+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:09.387 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.385+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.442 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.440+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:54:09.507 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.504+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.529 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.527+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:09.541 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.538+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.576 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.574+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:54:09.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.687+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.709 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.707+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:09.718 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.716+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.768 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.766+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:54:09.838 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.835+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.884 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.882+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:09.901 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.899+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:09.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.941+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:54:09.995 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:09.993+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.014 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.012+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:10.027 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.025+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.067 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.065+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:54:10.132 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.130+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.154 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.151+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:10.170 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.166+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.222 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.220+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:54:10.271 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.269+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.291 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.289+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:10.298 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.296+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.333 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.331+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:54:10.425 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.422+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.464 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.462+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:10.479 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.477+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.536 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.533+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:54:10.612 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.609+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.639 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.637+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:10.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.656+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.712 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.709+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:54:10.770 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.767+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.791 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.789+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:10.800 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.798+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.839 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.836+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:54:10.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.877+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.901+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:10.911 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.909+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:10.942 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.940+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:54:10.985 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:10.982+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.004 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.001+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:11.016 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.014+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.048 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.046+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:54:11.088 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.086+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.105 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.103+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:11.119 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.117+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.155 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.153+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:54:11.245 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.243+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.281 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.278+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:11.298 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.296+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.360 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.357+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:54:11.452 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.450+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.480 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.478+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:11.497 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.495+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.546 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.544+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:54:11.630 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.628+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.657 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.655+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:11.668 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.666+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.726 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.723+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:54:11.811 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.809+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.847 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.845+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:11.862 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.860+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.910 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.908+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:54:11.962 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.960+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:11.982 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.980+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:11.993 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:11.991+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:12.031 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.029+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:54:12.085 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.083+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:12.109 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.107+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:12.130 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.128+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:12.168 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.166+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:54:12.217 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.214+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:12.239 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.237+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:12.252 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.250+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:12.305 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.299+0000 7f799af96ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:54:12.361 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.359+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:12.380 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.378+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:12.392 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.390+0000 7f799af96ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:12.536 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.534+0000 7f799af96ec0 20 remove_watcher() i=0 2019-12-09T09:54:12.536 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.534+0000 7f799af96ec0 2 removed watcher, disabling cache 2019-12-09T09:54:12.575 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.573+0000 7f799af96ec0 20 remove_watcher() i=1 2019-12-09T09:54:12.619 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.617+0000 7f799af96ec0 20 remove_watcher() i=2 2019-12-09T09:54:12.679 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.677+0000 7f799af96ec0 20 remove_watcher() i=3 2019-12-09T09:54:12.732 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.730+0000 7f799af96ec0 20 remove_watcher() i=4 2019-12-09T09:54:12.779 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.776+0000 7f799af96ec0 20 remove_watcher() i=5 2019-12-09T09:54:12.832 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.830+0000 7f799af96ec0 20 remove_watcher() i=6 2019-12-09T09:54:12.875 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:12.873+0000 7f799af96ec0 20 remove_watcher() i=7 2019-12-09T09:54:12.887 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T09:54:13.017 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.015+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.042 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.040+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:13.043 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.040+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.051 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.049+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:13.052 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.049+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.054+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:13.056 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.054+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.065 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.063+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:54:13.312 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.310+0000 7fcaaf6e1ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T09:54:13.312 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.310+0000 7fcaaf6e1ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T09:54:13.312 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.310+0000 7fcaaf6e1ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:54:13.313 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.310+0000 7fcaaf6e1ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:54:13.313 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.310+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.327 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.325+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:54:13.328 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.325+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.345 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.343+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:54:13.345 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.343+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.359 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.356+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:54:13.359 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.356+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.375 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.373+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:54:13.375 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.373+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.388 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.385+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:13.388 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.385+0000 7fcaaf6e1ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:54:13.389 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.385+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.393 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.392+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:13.394 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.392+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.407 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.405+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:54:13.408 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.405+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.426 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.424+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:54:13.426 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.424+0000 7fcaaf6e1ec0 20 zonegroup default 2019-12-09T09:54:13.427 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.424+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.434 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.432+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:13.434 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.432+0000 7fcaaf6e1ec0 10 Cannot find current period zone using local zone 2019-12-09T09:54:13.434 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.432+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.443 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.441+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:54:13.443 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.441+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.457 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.455+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:54:13.457 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.455+0000 7fcaaf6e1ec0 20 rados->read ofs=0 len=0 2019-12-09T09:54:13.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.475+0000 7fcaaf6e1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:54:13.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.475+0000 7fcaaf6e1ec0 20 zone default found 2019-12-09T09:54:13.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.475+0000 7fcaaf6e1ec0 20 started sync module instance, tier type = 2019-12-09T09:54:13.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:13.475+0000 7fcaaf6e1ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T09:54:14.255 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.252+0000 7fcaaf6e1ec0 20 add_watcher() i=0 2019-12-09T09:54:14.255 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.252+0000 7fcaaf6e1ec0 20 add_watcher() i=1 2019-12-09T09:54:14.255 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.252+0000 7fcaaf6e1ec0 20 add_watcher() i=2 2019-12-09T09:54:14.255 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.252+0000 7fcaaf6e1ec0 20 add_watcher() i=3 2019-12-09T09:54:14.255 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.252+0000 7fcaaf6e1ec0 20 add_watcher() i=4 2019-12-09T09:54:14.255 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.252+0000 7fcaaf6e1ec0 20 add_watcher() i=5 2019-12-09T09:54:14.256 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.252+0000 7fcaaf6e1ec0 20 add_watcher() i=6 2019-12-09T09:54:14.301 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.299+0000 7fcaaf6e1ec0 20 add_watcher() i=7 2019-12-09T09:54:14.302 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.299+0000 7fcaaf6e1ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:54:14.302 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.299+0000 7fca6d7e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:54:14.309 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.302+0000 7fca9d10c700 0 -- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x56122c2c6520 msgr2=0x56122c2da550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T09:54:14.352 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:14.350+0000 7fca9c90b700 0 -- 172.21.15.35:0/3977034908 >> [v2:172.21.15.22:6800/8484,v1:172.21.15.22:6801/8484] conn(0x7fc9e0022480 msgr2=0x7fc9e0024990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:54:15.917 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:15.915+0000 7fc9df7fe700 20 reqs_thread_entry: start 2019-12-09T09:54:15.917 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:15.915+0000 7fc9dd7fa700 20 reqs_thread_entry: start 2019-12-09T09:54:15.918 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:15.915+0000 7fcaaf6e1ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:54:15.918 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:15.915+0000 7fc9cb7fe700 20 reqs_thread_entry: start 2019-12-09T09:54:15.918 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:15.916+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:54:16.006 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.004+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.051 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.049+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:16.074 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.072+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.126 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.124+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:54:16.221 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.219+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.269 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.267+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:16.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.295+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.362 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.360+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:54:16.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.431+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.463 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.461+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:16.484 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.482+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.530 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.528+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:54:16.590 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.588+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.611 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.609+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:16.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.620+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.650+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:54:16.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.706+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.741+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:16.762 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.760+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.812 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.810+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:54:16.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.876+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.901+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:16.913 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.911+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:16.947 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.945+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:54:16.995 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:16.993+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.017 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.015+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:17.025 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.024+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.068 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.066+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:54:17.114 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.112+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.131 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.129+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:17.143 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.141+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.179 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.177+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:54:17.225 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.223+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.241 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.239+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:17.251 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.249+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.282+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:54:17.329 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.327+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.352 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.349+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:17.364 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.362+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.398 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.396+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:54:17.481 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.479+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.519 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.517+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:17.531 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.529+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.590 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.588+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:54:17.669 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.667+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.691+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:17.705 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.703+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.749 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.747+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:54:17.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.832+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.856 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.854+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:17.869 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.867+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:17.921 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.912+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:54:17.983 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:17.981+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.008 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.006+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:18.039 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.037+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.072 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.070+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:54:18.112 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.110+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.127 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.125+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:18.137 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.135+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.173 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.170+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:54:18.228 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.226+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.249 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.247+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:18.261 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.259+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.305+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:54:18.361 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.359+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.382 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.380+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:18.399 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.397+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.442 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.440+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:54:18.508 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.505+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.535 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.533+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:18.548 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.546+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.595 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.592+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:54:18.646 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.644+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.664 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.662+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:18.673 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.671+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.701 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.699+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:54:18.746 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.742+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.761 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.759+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:18.772 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.769+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.809 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.807+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:54:18.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.855+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.876 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.874+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:18.890 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.888+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.922 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.919+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:54:18.972 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.970+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:18.992 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.990+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:19.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:18.999+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.045 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.043+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:54:19.095 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.093+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.115+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:19.127 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.125+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.158 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.156+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:54:19.219 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.217+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.240 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.238+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:19.251 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.249+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.287 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.284+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:54:19.363 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.360+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.388 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.385+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:19.399 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.397+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.431+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:54:19.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.475+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.495 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.493+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:19.508 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.506+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.548 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.546+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:54:19.641 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.639+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.679 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.676+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:19.699 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.697+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.758 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.756+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:54:19.860 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.858+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.892 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.890+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:19.912 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.910+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:19.966 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:19.963+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:54:20.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.031+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:20.053 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.051+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:20.067 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.065+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:20.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.112+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:54:20.186 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.184+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:20.208 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.206+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:20.218 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.216+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:20.254 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.252+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:54:20.304 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.302+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:20.334 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.332+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:20.356 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.354+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:20.392 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.390+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:54:20.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.442+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:20.462 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.460+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:54:20.473 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.471+0000 7fcaaf6e1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:54:20.581 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.579+0000 7fcaaf6e1ec0 20 remove_watcher() i=0 2019-12-09T09:54:20.581 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.579+0000 7fcaaf6e1ec0 2 removed watcher, disabling cache 2019-12-09T09:54:20.634 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.632+0000 7fcaaf6e1ec0 20 remove_watcher() i=1 2019-12-09T09:54:20.680 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.678+0000 7fcaaf6e1ec0 20 remove_watcher() i=2 2019-12-09T09:54:20.765 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.763+0000 7fcaaf6e1ec0 20 remove_watcher() i=3 2019-12-09T09:54:20.837 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.835+0000 7fcaaf6e1ec0 20 remove_watcher() i=4 2019-12-09T09:54:20.889 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.887+0000 7fcaaf6e1ec0 20 remove_watcher() i=5 2019-12-09T09:54:20.943 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.941+0000 7fcaaf6e1ec0 20 remove_watcher() i=6 2019-12-09T09:54:20.997 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T09:54:20.989+0000 7fcaaf6e1ec0 20 remove_watcher() i=7 2019-12-09T09:54:21.003 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2019-12-09T09:54:21.724 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :clean 2019-12-09T09:54:22.221 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :compileJava 2019-12-09T09:54:22.222 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :processResources 2019-12-09T09:54:22.222 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :classes 2019-12-09T09:54:22.426 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:54:22.426 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :compileTestJava 2019-12-09T09:54:22.426 INFO:teuthology.orchestra.run.smithi035.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T09:54:22.426 INFO:teuthology.orchestra.run.smithi035.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T09:54:22.426 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:54:22.426 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T09:54:22.427 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :testClasses 2019-12-09T09:54:24.921 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:54:24.921 INFO:teuthology.orchestra.run.smithi035.stdout:> Task :test 2019-12-09T09:54:24.921 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:54:24.921 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2019-12-09T09:54:29.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:54:29.421 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2019-12-09T09:54:36.520 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:54:36.520 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2019-12-09T09:54:39.120 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:54:39.121 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2019-12-09T09:54:45.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:54:45.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2019-12-09T09:54:47.120 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:54:47.120 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2019-12-09T09:54:49.094 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:42:25.357 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:49.101 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:42:25.364 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:49.108 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:42:25.331 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:49.114 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:42:25.337 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:49.152 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:42:25.375 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:49.156 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:42:25.379 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:49.301 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:42:25.564 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:49.305 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:42:25.528 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:49.359 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:42:25.582 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:50.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:54:50.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2019-12-09T09:55:04.177 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:42:40.440 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:55:04.185 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:42:40.448 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:55:04.216 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:42:40.479 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T09:55:04.217 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:42:40.440 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:55:04.221 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:42:40.444 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:55:04.228 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:42:40.451 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:55:04.260 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:42:40.483 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:55:04.289 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:42:40.512 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T09:55:04.300 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:42:40.522 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T09:55:10.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:10.621 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2019-12-09T09:55:17.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:17.720 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2019-12-09T09:55:19.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:19.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2019-12-09T09:55:26.120 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:26.120 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2019-12-09T09:55:29.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:29.221 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2019-12-09T09:55:37.520 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:37.520 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2019-12-09T09:55:37.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:37.721 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2019-12-09T09:55:38.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:38.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2019-12-09T09:55:39.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:39.921 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2019-12-09T09:55:44.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:44.720 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2019-12-09T09:55:50.324 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:50.324 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2019-12-09T09:55:56.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:55:56.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2019-12-09T09:56:00.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:00.421 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2019-12-09T09:56:07.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:07.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2019-12-09T09:56:09.921 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:09.921 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2019-12-09T09:56:18.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:18.920 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2019-12-09T09:56:22.020 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:22.020 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2019-12-09T09:56:28.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:28.920 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2019-12-09T09:56:32.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:32.421 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2019-12-09T09:56:38.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:38.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2019-12-09T09:56:40.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:40.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2019-12-09T09:56:46.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:46.920 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2019-12-09T09:56:49.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:49.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2019-12-09T09:56:54.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:56:54.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2019-12-09T09:57:01.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:57:01.321 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2019-12-09T09:57:09.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:57:09.821 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2019-12-09T09:57:48.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:57:48.921 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2019-12-09T09:58:06.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T09:58:06.621 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2019-12-09T10:00:00.481 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:47:36.744 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:00.499 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:47:36.763 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:00.507 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:47:36.730 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:00.512 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:47:36.735 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:00.515 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:47:36.738 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:00.517 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:47:36.741 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:00.727 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:47:36.950 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:00.749 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:47:37.012 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:00.773 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:47:36.996 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:02:03.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:02:03.721 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2019-12-09T10:02:21.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:02:21.420 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2019-12-09T10:02:25.020 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:02:25.020 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2019-12-09T10:02:32.020 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:02:32.020 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2019-12-09T10:02:38.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:02:38.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2019-12-09T10:02:50.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:02:50.920 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2019-12-09T10:03:08.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:03:08.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2019-12-09T10:03:16.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:03:16.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2019-12-09T10:03:17.020 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:03:17.020 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2019-12-09T10:03:18.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:03:18.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2019-12-09T10:03:26.253 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:51:02.517 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:03:26.257 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:51:02.521 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:03:26.316 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:51:02.539 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:03:26.318 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:51:02.541 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:03:26.329 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:51:02.552 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:03:26.345 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:51:02.568 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:03:26.445 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:51:02.668 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:03:26.461 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:51:02.724 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:03:26.517 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:51:02.739 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:03:38.285 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T10:03:38.206+0000 41a65700 -1 handle_data failed with exit code -5 2019-12-09T10:03:39.321 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:03:39.321 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2019-12-09T10:03:50.121 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:03:50.121 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2019-12-09T10:04:08.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:04:08.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2019-12-09T10:04:23.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:04:23.920 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2019-12-09T10:04:41.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:04:41.821 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2019-12-09T10:04:47.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:04:47.821 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2019-12-09T10:05:11.299 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:52:47.562 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:05:11.309 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:52:47.573 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:05:11.325 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:52:47.548 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:05:11.331 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:52:47.554 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:05:11.442 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:52:47.665 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:05:11.449 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:52:47.713 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:05:11.454 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:52:47.677 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:05:11.457 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:52:47.680 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:05:11.564 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:52:47.787 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:06:46.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:06:46.321 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2019-12-09T10:06:52.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:06:52.420 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2019-12-09T10:07:25.520 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:07:25.520 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2019-12-09T10:07:35.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:07:35.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2019-12-09T10:08:03.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:08:03.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2019-12-09T10:08:11.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:08:11.720 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2019-12-09T10:08:18.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:08:18.720 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2019-12-09T10:08:29.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:08:29.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2019-12-09T10:08:29.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:08:29.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2019-12-09T10:08:30.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:08:30.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative STARTED 2019-12-09T10:08:31.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:08:31.920 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative PASSED 2019-12-09T10:09:37.120 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:09:37.120 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2019-12-09T10:09:40.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:09:40.920 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2019-12-09T10:09:48.120 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:09:48.120 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2019-12-09T10:09:51.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:09:51.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2019-12-09T10:09:57.020 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:09:57.020 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2019-12-09T10:09:59.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:09:59.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2019-12-09T10:10:06.020 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:10:06.020 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2019-12-09T10:10:08.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:10:08.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2019-12-09T10:10:14.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:10:14.420 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2019-12-09T10:10:16.020 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:10:16.020 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2019-12-09T10:10:22.120 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:10:22.120 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2019-12-09T10:10:24.205 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:58:00.469 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:10:24.222 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:58:00.445 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:10:24.224 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:58:00.488 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:10:24.226 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:58:00.449 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:10:24.233 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:58:00.456 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:10:24.238 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:58:00.461 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:10:24.429 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:58:00.652 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:10:24.487 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:58:00.710 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:10:24.506 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:58:00.770 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:10:25.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:10:25.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2019-12-09T10:10:32.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:10:32.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2019-12-09T10:10:42.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:10:42.420 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2019-12-09T10:10:51.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:10:51.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2019-12-09T10:11:02.120 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:11:02.120 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2019-12-09T10:11:14.120 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:11:14.120 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2019-12-09T10:11:22.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:11:22.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2019-12-09T10:11:35.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:11:35.421 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2019-12-09T10:11:42.120 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:11:42.120 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2019-12-09T10:11:48.583 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:59:24.847 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:48.589 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:59:24.854 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:48.591 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:59:24.814 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:48.608 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:59:24.831 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:48.610 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:59:24.834 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:48.614 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:59:24.837 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:48.665 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:00:59:24.929 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:48.692 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:00:59:24.915 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:48.692 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:00:59:24.915 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:55.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:11:55.920 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2019-12-09T10:12:05.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:12:05.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2019-12-09T10:12:16.520 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:12:16.520 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2019-12-09T10:12:24.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:12:24.720 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2019-12-09T10:12:38.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:12:38.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2019-12-09T10:12:46.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:12:46.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2019-12-09T10:12:57.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:12:57.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2019-12-09T10:13:06.120 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:13:06.120 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2019-12-09T10:13:17.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:13:17.920 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2019-12-09T10:13:24.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:13:24.920 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2019-12-09T10:13:36.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:13:36.421 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2019-12-09T10:13:43.821 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:13:43.821 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2019-12-09T10:13:56.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:13:56.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2019-12-09T10:14:03.421 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:14:03.421 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2019-12-09T10:14:14.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:14:14.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2019-12-09T10:14:21.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:14:21.420 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2019-12-09T10:14:30.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:14:30.321 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2019-12-09T10:14:35.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:14:35.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2019-12-09T10:14:44.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:14:44.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2019-12-09T10:14:49.520 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:14:49.520 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2019-12-09T10:15:01.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:15:01.720 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2019-12-09T10:15:06.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:15:06.720 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2019-12-09T10:15:18.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:15:18.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2019-12-09T10:15:24.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:15:24.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2019-12-09T10:15:35.561 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:03:11.826 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:35.571 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:03:11.835 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:35.574 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:03:11.797 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:35.574 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:03:11.797 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:35.576 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:03:11.799 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:35.576 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:03:11.800 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:35.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:15:35.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2019-12-09T10:15:35.718 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:03:11.941 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:35.778 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:03:12.042 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:35.810 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:03:12.033 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:40.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:15:40.420 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2019-12-09T10:15:49.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:15:49.720 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2019-12-09T10:15:56.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:15:56.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2019-12-09T10:16:08.020 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:16:08.020 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2019-12-09T10:16:15.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:16:15.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2019-12-09T10:16:29.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:16:29.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2019-12-09T10:16:36.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:16:36.420 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2019-12-09T10:16:48.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:16:48.720 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2019-12-09T10:16:55.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:16:55.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2019-12-09T10:17:06.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:17:06.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2019-12-09T10:17:13.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:17:13.420 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2019-12-09T10:17:23.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:17:23.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2019-12-09T10:17:27.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:17:27.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2019-12-09T10:17:34.120 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:17:34.120 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2019-12-09T10:17:38.520 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:17:38.520 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2019-12-09T10:17:44.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:17:44.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2019-12-09T10:17:49.920 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:17:49.920 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2019-12-09T10:17:58.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:17:58.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2019-12-09T10:18:01.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:18:01.420 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2019-12-09T10:18:09.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:18:09.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2019-12-09T10:18:19.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:18:19.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2019-12-09T10:18:26.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:18:26.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2019-12-09T10:18:26.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:18:26.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2019-12-09T10:18:28.020 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:18:28.020 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2019-12-09T10:18:31.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:18:31.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2019-12-09T10:18:40.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:18:40.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2019-12-09T10:18:44.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:18:44.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2019-12-09T10:18:53.620 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:18:53.620 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2019-12-09T10:18:56.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:18:56.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2019-12-09T10:19:04.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:19:04.720 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2019-12-09T10:19:07.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:19:07.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2019-12-09T10:19:13.820 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:19:13.820 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent STARTED 2019-12-09T10:19:18.520 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:19:18.520 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent PASSED 2019-12-09T10:19:26.020 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:19:26.020 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2019-12-09T10:19:43.320 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:19:43.320 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2019-12-09T10:19:51.220 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:19:51.220 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI STARTED 2019-12-09T10:20:10.383 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:07:46.648 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:10.406 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:07:46.671 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:10.428 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:07:46.651 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:10.429 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:07:46.653 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:10.430 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:07:46.653 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:10.435 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:07:46.658 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:10.531 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:07:46.795 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:10.548 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:07:46.772 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:10.570 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:07:46.793 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:20.520 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:20:20.520 INFO:teuthology.orchestra.run.smithi035.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI PASSED 2019-12-09T10:20:29.165 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:20:29.165 INFO:teuthology.orchestra.run.smithi035.stdout:---------------------------------------------------------------------- 2019-12-09T10:20:29.165 INFO:teuthology.orchestra.run.smithi035.stdout:| Results: SUCCESS (72 tests, 72 successes, 0 failures, 0 skipped) | 2019-12-09T10:20:29.166 INFO:teuthology.orchestra.run.smithi035.stdout:---------------------------------------------------------------------- 2019-12-09T10:20:29.166 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:20:29.166 INFO:teuthology.orchestra.run.smithi035.stdout:BUILD SUCCESSFUL in 26m 8s 2019-12-09T10:20:29.166 INFO:teuthology.orchestra.run.smithi035.stdout:5 actionable tasks: 5 executed 2019-12-09T10:20:29.172 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T10:20:29.310 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.308+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.333 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.331+0000 7f007dabeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:29.333 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.331+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.347+0000 7f007dabeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:29.350 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.347+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.354 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.352+0000 7f007dabeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:29.354 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.352+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.361 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.360+0000 7f007dabeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:29.573 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.571+0000 7f007dabeec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T10:20:29.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.571+0000 7f007dabeec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T10:20:29.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.571+0000 7f007dabeec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:20:29.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.571+0000 7f007dabeec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:20:29.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.571+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.584 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.583+0000 7f007dabeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:29.585 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.583+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.597 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.595+0000 7f007dabeec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:20:29.597 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.595+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.618 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.616+0000 7f007dabeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:29.618 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.616+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.629 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.626+0000 7f007dabeec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:20:29.629 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.627+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.640 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.637+0000 7f007dabeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:29.640 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.637+0000 7f007dabeec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:20:29.640 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.637+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.646 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.644+0000 7f007dabeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:29.646 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.644+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.655 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.653+0000 7f007dabeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:29.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.653+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.673 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.671+0000 7f007dabeec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:20:29.673 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.671+0000 7f007dabeec0 20 zonegroup default 2019-12-09T10:20:29.673 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.671+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.678 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.676+0000 7f007dabeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:29.678 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.676+0000 7f007dabeec0 10 Cannot find current period zone using local zone 2019-12-09T10:20:29.678 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.676+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.683+0000 7f007dabeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:29.685 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.683+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.694 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.692+0000 7f007dabeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:29.694 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.692+0000 7f007dabeec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:29.706 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.704+0000 7f007dabeec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:20:29.707 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.704+0000 7f007dabeec0 20 zone default found 2019-12-09T10:20:29.707 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.704+0000 7f007dabeec0 20 started sync module instance, tier type = 2019-12-09T10:20:29.707 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:29.704+0000 7f007dabeec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T10:20:30.110 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.109+0000 7f007dabeec0 20 add_watcher() i=0 2019-12-09T10:20:30.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.109+0000 7f007dabeec0 20 add_watcher() i=1 2019-12-09T10:20:30.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.109+0000 7f007dabeec0 20 add_watcher() i=2 2019-12-09T10:20:30.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.109+0000 7f007dabeec0 20 add_watcher() i=3 2019-12-09T10:20:30.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.109+0000 7f007dabeec0 20 add_watcher() i=4 2019-12-09T10:20:30.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.109+0000 7f007dabeec0 20 add_watcher() i=5 2019-12-09T10:20:30.112 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.109+0000 7f007dabeec0 20 add_watcher() i=6 2019-12-09T10:20:30.148 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.146+0000 7f007dabeec0 20 add_watcher() i=7 2019-12-09T10:20:30.149 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.146+0000 7f007dabeec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:20:30.149 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.147+0000 7f003b7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:20:30.165 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.163+0000 7f006ace8700 0 -- 172.21.15.35:0/3792049282 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x55fabf6d3f50 msgr2=0x55fabf6d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:20:30.199 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:30.198+0000 7f006ace8700 0 -- 172.21.15.35:0/3183930393 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7effac002f90 msgr2=0x7effac0033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:20:31.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:31.942+0000 7effa9ffb700 20 reqs_thread_entry: start 2019-12-09T10:20:31.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:31.942+0000 7effa37fe700 20 reqs_thread_entry: start 2019-12-09T10:20:31.945 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:31.942+0000 7f007dabeec0 20 init_complete bucket index max shards: 0 2019-12-09T10:20:31.945 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:31.943+0000 7effa1ffb700 20 reqs_thread_entry: start 2019-12-09T10:20:31.945 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:31.943+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:20:32.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.008+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:32.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.044+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:32.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T10:20:32.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='2~8M3qYb-tDc9oLt4fnSlaox-ovevJd_x', time=2019-12-09T12:04:43.960941+0000, chain.objs.size()=9 2019-12-09T10:20:32.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.74__multipart_key1.2~8M3qYb-tDc9oLt4fnSlaox-ovevJd_x.7 2019-12-09T10:20:32.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.74__shadow_key1.2~8M3qYb-tDc9oLt4fnSlaox-ovevJd_x.7_1 2019-12-09T10:20:32.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.74__multipart_key1.2~8M3qYb-tDc9oLt4fnSlaox-ovevJd_x.8 2019-12-09T10:20:32.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.74__shadow_key1.2~8M3qYb-tDc9oLt4fnSlaox-ovevJd_x.8_1 2019-12-09T10:20:32.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.74__multipart_key1.2~8M3qYb-tDc9oLt4fnSlaox-ovevJd_x.9 2019-12-09T10:20:32.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.74__shadow_key1.2~8M3qYb-tDc9oLt4fnSlaox-ovevJd_x.9_1 2019-12-09T10:20:32.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.74__multipart_key1.2~8M3qYb-tDc9oLt4fnSlaox-ovevJd_x.10 2019-12-09T10:20:32.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.74__shadow_key1.2~8M3qYb-tDc9oLt4fnSlaox-ovevJd_x.10_1 2019-12-09T10:20:32.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.74__multipart_key1.2~8M3qYb-tDc9oLt4fnSlaox-ovevJd_x.11 2019-12-09T10:20:32.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1248', time=2019-12-09T12:06:47.512389+0000, chain.objs.size()=7 2019-12-09T10:20:32.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~yeroJKuKAcKTsm6Cj42jaE28g3LJypa.1 2019-12-09T10:20:32.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~yeroJKuKAcKTsm6Cj42jaE28g3LJypa.1_1 2019-12-09T10:20:32.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.072+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~yeroJKuKAcKTsm6Cj42jaE28g3LJypa.1_2 2019-12-09T10:20:32.323 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.321+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~yeroJKuKAcKTsm6Cj42jaE28g3LJypa.2 2019-12-09T10:20:32.324 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.321+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~yeroJKuKAcKTsm6Cj42jaE28g3LJypa.2_1 2019-12-09T10:20:32.324 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.321+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~yeroJKuKAcKTsm6Cj42jaE28g3LJypa.2_2 2019-12-09T10:20:32.324 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.321+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~yeroJKuKAcKTsm6Cj42jaE28g3LJypa.3 2019-12-09T10:20:32.324 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.321+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1897', time=2019-12-09T12:19:46.512322+0000, chain.objs.size()=9 2019-12-09T10:20:32.324 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.321+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.118__multipart_key1.2~iaVZ7ISB3zAzwys_reZhNIsPQfiiAks.1 2019-12-09T10:20:32.324 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.321+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.118__shadow_key1.2~iaVZ7ISB3zAzwys_reZhNIsPQfiiAks.1_1 2019-12-09T10:20:32.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.321+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.118__multipart_key1.2~iaVZ7ISB3zAzwys_reZhNIsPQfiiAks.2 2019-12-09T10:20:32.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.321+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.118__shadow_key1.2~iaVZ7ISB3zAzwys_reZhNIsPQfiiAks.2_1 2019-12-09T10:20:32.499 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.497+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.118__multipart_key1.2~iaVZ7ISB3zAzwys_reZhNIsPQfiiAks.3 2019-12-09T10:20:32.499 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.497+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.118__shadow_key1.2~iaVZ7ISB3zAzwys_reZhNIsPQfiiAks.3_1 2019-12-09T10:20:32.499 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.497+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.118__multipart_key1.2~iaVZ7ISB3zAzwys_reZhNIsPQfiiAks.4 2019-12-09T10:20:32.499 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.497+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.118__shadow_key1.2~iaVZ7ISB3zAzwys_reZhNIsPQfiiAks.4_1 2019-12-09T10:20:32.604 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.602+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.118__multipart_key1.2~iaVZ7ISB3zAzwys_reZhNIsPQfiiAks.5 2019-12-09T10:20:32.713 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.711+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:32.798 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.796+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:20:32.889 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.887+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:32.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.925+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:32.954 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.951+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:20:32.954 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.951+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1043', time=2019-12-09T11:57:52.859666+0000, chain.objs.size()=9 2019-12-09T10:20:32.954 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.951+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__multipart_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.1 2019-12-09T10:20:32.954 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.951+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__shadow_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.1_1 2019-12-09T10:20:32.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.951+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__multipart_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.2 2019-12-09T10:20:32.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.952+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__shadow_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.2_1 2019-12-09T10:20:32.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.952+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__multipart_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.3 2019-12-09T10:20:32.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.952+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__shadow_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.3_1 2019-12-09T10:20:32.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.952+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__multipart_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.4 2019-12-09T10:20:32.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.952+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__shadow_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.4_1 2019-12-09T10:20:32.956 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:32.952+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__multipart_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.5 2019-12-09T10:20:33.135 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.133+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:33.231 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.229+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:20:33.278 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.276+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:33.296 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.294+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:33.305 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.303+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:33.341 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.339+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:20:33.398 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.396+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:33.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.420+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:33.438 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.436+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:33.485 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.483+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:20:33.547 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.545+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:33.575 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.573+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:33.586 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.584+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:33.629 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.627+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:20:33.698 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.696+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:33.722 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.720+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:33.731 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.729+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:33.774 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.772+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:20:33.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.839+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:33.871 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.869+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:33.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.886+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:33.944 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:33.942+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:20:34.050 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.048+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:34.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.074+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:34.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:20:34.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1100', time=2019-12-09T12:02:14.731301+0000, chain.objs.size()=7 2019-12-09T10:20:34.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~_rNp82xSMFimelfjTtbplyevacKHnh2.1 2019-12-09T10:20:34.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~_rNp82xSMFimelfjTtbplyevacKHnh2.1_1 2019-12-09T10:20:34.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~_rNp82xSMFimelfjTtbplyevacKHnh2.1_2 2019-12-09T10:20:34.116 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~_rNp82xSMFimelfjTtbplyevacKHnh2.2 2019-12-09T10:20:34.116 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~_rNp82xSMFimelfjTtbplyevacKHnh2.2_1 2019-12-09T10:20:34.116 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~_rNp82xSMFimelfjTtbplyevacKHnh2.2_2 2019-12-09T10:20:34.116 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~_rNp82xSMFimelfjTtbplyevacKHnh2.3 2019-12-09T10:20:34.116 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4', time=2019-12-09T12:08:24.420288+0000, chain.objs.size()=34 2019-12-09T10:20:34.116 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.1 2019-12-09T10:20:34.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.2 2019-12-09T10:20:34.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.3 2019-12-09T10:20:34.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.4 2019-12-09T10:20:34.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.112+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.5 2019-12-09T10:20:34.418 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.416+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.6 2019-12-09T10:20:34.418 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.416+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.7 2019-12-09T10:20:34.419 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.416+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.8 2019-12-09T10:20:34.419 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.416+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.9 2019-12-09T10:20:34.419 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.416+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.10 2019-12-09T10:20:34.425 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.423+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.11 2019-12-09T10:20:34.426 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.423+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.12 2019-12-09T10:20:34.472 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.470+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.13 2019-12-09T10:20:34.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.507+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.14 2019-12-09T10:20:34.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.507+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.15 2019-12-09T10:20:34.614 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.612+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.16 2019-12-09T10:20:34.614 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.612+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.17 2019-12-09T10:20:34.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.654+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.CuJAv9VOZqn-8BM8bTBBlqrBXoa3GUD.18 2019-12-09T10:20:34.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.654+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.nYYIVu8ThEd9CASlcgI_C5Bxk5ug9dm.19 2019-12-09T10:20:34.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.654+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.Kn8URHcl53df9OFU0HRvHvbxJeuc5SL.20 2019-12-09T10:20:34.663 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.661+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.21 2019-12-09T10:20:34.732 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.730+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.22 2019-12-09T10:20:34.732 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.730+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.23 2019-12-09T10:20:34.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.839+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.24 2019-12-09T10:20:34.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.839+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.25 2019-12-09T10:20:34.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.839+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.26 2019-12-09T10:20:34.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.839+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.27 2019-12-09T10:20:34.842 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.839+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.28 2019-12-09T10:20:34.842 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.839+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.29 2019-12-09T10:20:34.842 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.839+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.30 2019-12-09T10:20:34.871 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.869+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.31 2019-12-09T10:20:34.916 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.914+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.32 2019-12-09T10:20:34.939 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.937+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.33 2019-12-09T10:20:34.939 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:34.937+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.78__multipart_key1.2~bTeUjwNc7pdg_4RtqHxxDKoqvjMvbI4.34 2019-12-09T10:20:35.153 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.151+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:35.288 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.286+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:20:35.339 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.337+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:35.359 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.357+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:35.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:20:35.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1219', time=2019-12-09T12:05:33.628275+0000, chain.objs.size()=9 2019-12-09T10:20:35.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~0iN1MifWgV6HPvfC0qXOYsSCi5YHSwz.1 2019-12-09T10:20:35.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~0iN1MifWgV6HPvfC0qXOYsSCi5YHSwz.1_1 2019-12-09T10:20:35.386 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~0iN1MifWgV6HPvfC0qXOYsSCi5YHSwz.2 2019-12-09T10:20:35.387 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~0iN1MifWgV6HPvfC0qXOYsSCi5YHSwz.2_1 2019-12-09T10:20:35.387 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~0iN1MifWgV6HPvfC0qXOYsSCi5YHSwz.3 2019-12-09T10:20:35.387 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~0iN1MifWgV6HPvfC0qXOYsSCi5YHSwz.3_1 2019-12-09T10:20:35.387 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~0iN1MifWgV6HPvfC0qXOYsSCi5YHSwz.4 2019-12-09T10:20:35.387 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~0iN1MifWgV6HPvfC0qXOYsSCi5YHSwz.4_1 2019-12-09T10:20:35.387 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~0iN1MifWgV6HPvfC0qXOYsSCi5YHSwz.5 2019-12-09T10:20:35.387 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1278', time=2019-12-09T12:08:05.989161+0000, chain.objs.size()=9 2019-12-09T10:20:35.388 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.77__multipart_key1.2~ddQpyIu5yzBnucejkucr_b9vqE0CGl8.1 2019-12-09T10:20:35.388 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.77__shadow_key1.2~ddQpyIu5yzBnucejkucr_b9vqE0CGl8.1_1 2019-12-09T10:20:35.388 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.383+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.77__multipart_key1.2~ddQpyIu5yzBnucejkucr_b9vqE0CGl8.2 2019-12-09T10:20:35.557 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.555+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.77__shadow_key1.2~ddQpyIu5yzBnucejkucr_b9vqE0CGl8.2_1 2019-12-09T10:20:35.598 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.597+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.77__multipart_key1.2~ddQpyIu5yzBnucejkucr_b9vqE0CGl8.3 2019-12-09T10:20:35.599 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.597+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.77__shadow_key1.2~ddQpyIu5yzBnucejkucr_b9vqE0CGl8.3_1 2019-12-09T10:20:35.637 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.635+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.77__multipart_key1.2~ddQpyIu5yzBnucejkucr_b9vqE0CGl8.4 2019-12-09T10:20:35.637 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.635+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.77__shadow_key1.2~ddQpyIu5yzBnucejkucr_b9vqE0CGl8.4_1 2019-12-09T10:20:35.644 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.642+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.77__multipart_key1.2~ddQpyIu5yzBnucejkucr_b9vqE0CGl8.5 2019-12-09T10:20:35.824 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.822+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:35.907 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.905+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:20:35.949 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.947+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:35.965 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.963+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:35.974 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:35.972+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:36.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.031+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:20:36.095 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.093+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:36.117 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.115+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:36.128 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.126+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:36.169 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.167+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:20:36.250 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.248+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:36.278 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.276+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:36.293 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.291+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:36.355 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.353+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:20:36.427 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.425+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:36.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.454+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:36.489 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.486+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:20:36.489 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.486+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1156', time=2019-12-09T12:03:43.018357+0000, chain.objs.size()=9 2019-12-09T10:20:36.489 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.486+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.72__multipart_key1.2~Olzs4gJHnnACkIDjZ9Pz6NlZ-Ucdur7.1 2019-12-09T10:20:36.489 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.487+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.72__shadow_key1.2~Olzs4gJHnnACkIDjZ9Pz6NlZ-Ucdur7.1_1 2019-12-09T10:20:36.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.487+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.72__multipart_key1.2~Olzs4gJHnnACkIDjZ9Pz6NlZ-Ucdur7.2 2019-12-09T10:20:36.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.487+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.72__shadow_key1.2~Olzs4gJHnnACkIDjZ9Pz6NlZ-Ucdur7.2_1 2019-12-09T10:20:36.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.487+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.72__multipart_key1.2~Olzs4gJHnnACkIDjZ9Pz6NlZ-Ucdur7.3 2019-12-09T10:20:36.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.487+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.72__shadow_key1.2~Olzs4gJHnnACkIDjZ9Pz6NlZ-Ucdur7.3_1 2019-12-09T10:20:36.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.487+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.72__multipart_key1.2~Olzs4gJHnnACkIDjZ9Pz6NlZ-Ucdur7.4 2019-12-09T10:20:36.490 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.487+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.72__shadow_key1.2~Olzs4gJHnnACkIDjZ9Pz6NlZ-Ucdur7.4_1 2019-12-09T10:20:36.491 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.487+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.72__multipart_key1.2~Olzs4gJHnnACkIDjZ9Pz6NlZ-Ucdur7.5 2019-12-09T10:20:36.754 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.753+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:36.829 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.827+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:20:36.873 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.871+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:36.889 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.887+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:36.907 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.904+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T10:20:36.907 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.904+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.974', time=2019-12-09T11:55:51.895124+0000, chain.objs.size()=2 2019-12-09T10:20:36.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.56__shadow_.MSKQJmWx8FEdrABP76UlursflGFUrqy_1 2019-12-09T10:20:36.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.56__shadow_.MSKQJmWx8FEdrABP76UlursflGFUrqy_2 2019-12-09T10:20:36.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1058', time=2019-12-09T12:02:06.405003+0000, chain.objs.size()=5 2019-12-09T10:20:36.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.66__shadow_.LwF7zyPRpPaQrdod5jQsfWGphJHeVLw_1 2019-12-09T10:20:36.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.66__shadow_.LwF7zyPRpPaQrdod5jQsfWGphJHeVLw_2 2019-12-09T10:20:36.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.66__shadow_.LwF7zyPRpPaQrdod5jQsfWGphJHeVLw_3 2019-12-09T10:20:36.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.66__shadow_.LwF7zyPRpPaQrdod5jQsfWGphJHeVLw_4 2019-12-09T10:20:36.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.66__shadow_.LwF7zyPRpPaQrdod5jQsfWGphJHeVLw_5 2019-12-09T10:20:36.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1913', time=2019-12-09T12:20:22.551329+0000, chain.objs.size()=9 2019-12-09T10:20:36.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.119__multipart_key1/data/file.mpg.2~PhHMtPE4tdm_2PIn8P2CgVVXY_kNljP.1 2019-12-09T10:20:36.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.119__shadow_key1/data/file.mpg.2~PhHMtPE4tdm_2PIn8P2CgVVXY_kNljP.1_1 2019-12-09T10:20:36.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.119__multipart_key1/data/file.mpg.2~PhHMtPE4tdm_2PIn8P2CgVVXY_kNljP.2 2019-12-09T10:20:36.909 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.119__shadow_key1/data/file.mpg.2~PhHMtPE4tdm_2PIn8P2CgVVXY_kNljP.2_1 2019-12-09T10:20:36.910 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:36.905+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.119__multipart_key1/data/file.mpg.2~PhHMtPE4tdm_2PIn8P2CgVVXY_kNljP.3 2019-12-09T10:20:37.143 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.141+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.119__shadow_key1/data/file.mpg.2~PhHMtPE4tdm_2PIn8P2CgVVXY_kNljP.3_1 2019-12-09T10:20:37.143 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.141+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.119__multipart_key1/data/file.mpg.2~PhHMtPE4tdm_2PIn8P2CgVVXY_kNljP.4 2019-12-09T10:20:37.143 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.141+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.119__shadow_key1/data/file.mpg.2~PhHMtPE4tdm_2PIn8P2CgVVXY_kNljP.4_1 2019-12-09T10:20:37.143 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.141+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.119__multipart_key1/data/file.mpg.2~PhHMtPE4tdm_2PIn8P2CgVVXY_kNljP.5 2019-12-09T10:20:37.256 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.255+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:37.335 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.333+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:20:37.385 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.383+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:37.402 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.400+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:37.411 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.409+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:37.439 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.438+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:20:37.493 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.491+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:37.514 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.513+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:37.532 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.530+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:20:37.532 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.530+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1188', time=2019-12-09T12:04:13.776022+0000, chain.objs.size()=9 2019-12-09T10:20:37.533 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.530+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__multipart_file2.mpg.2~wEqRtUOW-NBJbwtRm35Augz4hCB7zF5.1 2019-12-09T10:20:37.533 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.530+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__shadow_file2.mpg.2~wEqRtUOW-NBJbwtRm35Augz4hCB7zF5.1_1 2019-12-09T10:20:37.533 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.530+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__multipart_file2.mpg.2~wEqRtUOW-NBJbwtRm35Augz4hCB7zF5.2 2019-12-09T10:20:37.533 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.530+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__shadow_file2.mpg.2~wEqRtUOW-NBJbwtRm35Augz4hCB7zF5.2_1 2019-12-09T10:20:37.533 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.530+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__multipart_file2.mpg.2~wEqRtUOW-NBJbwtRm35Augz4hCB7zF5.3 2019-12-09T10:20:37.533 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.530+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__shadow_file2.mpg.2~wEqRtUOW-NBJbwtRm35Augz4hCB7zF5.3_1 2019-12-09T10:20:37.534 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.530+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__multipart_file2.mpg.2~wEqRtUOW-NBJbwtRm35Augz4hCB7zF5.4 2019-12-09T10:20:37.534 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.530+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__shadow_file2.mpg.2~wEqRtUOW-NBJbwtRm35Augz4hCB7zF5.4_1 2019-12-09T10:20:37.534 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.530+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__multipart_file2.mpg.2~wEqRtUOW-NBJbwtRm35Augz4hCB7zF5.5 2019-12-09T10:20:37.771 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.769+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:37.848 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.846+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:20:37.891 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.889+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:37.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.906+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:37.920 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.918+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:37.955 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:37.953+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:20:38.040 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.038+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:38.071 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.068+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:38.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.084+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:38.154 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.152+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:20:38.248 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.246+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:38.288 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.286+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:38.302 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.301+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:38.345 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.344+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:20:38.401 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.399+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:38.434 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.433+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:38.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.463+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:20:38.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.463+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1066', time=2019-12-09T11:59:48.419971+0000, chain.objs.size()=9 2019-12-09T10:20:38.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.463+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~6CGnC2L22gmhTOhTFnbdEjTKG3jDXlS.1 2019-12-09T10:20:38.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.463+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~6CGnC2L22gmhTOhTFnbdEjTKG3jDXlS.1_1 2019-12-09T10:20:38.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.463+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~6CGnC2L22gmhTOhTFnbdEjTKG3jDXlS.2 2019-12-09T10:20:38.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.463+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~6CGnC2L22gmhTOhTFnbdEjTKG3jDXlS.2_1 2019-12-09T10:20:38.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.463+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~6CGnC2L22gmhTOhTFnbdEjTKG3jDXlS.3 2019-12-09T10:20:38.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.463+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~6CGnC2L22gmhTOhTFnbdEjTKG3jDXlS.3_1 2019-12-09T10:20:38.467 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.463+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~6CGnC2L22gmhTOhTFnbdEjTKG3jDXlS.4 2019-12-09T10:20:38.467 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.463+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~6CGnC2L22gmhTOhTFnbdEjTKG3jDXlS.4_1 2019-12-09T10:20:38.467 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.463+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~6CGnC2L22gmhTOhTFnbdEjTKG3jDXlS.5 2019-12-09T10:20:38.731 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.729+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:38.924 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.922+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:20:38.993 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:38.991+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:39.013 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.011+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:39.032 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:20:39.032 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1074', time=2019-12-09T12:00:29.691700+0000, chain.objs.size()=9 2019-12-09T10:20:39.032 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~yT6a3kE0M8BBcoBgnhJh2vzOCNKOn41.1 2019-12-09T10:20:39.032 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~yT6a3kE0M8BBcoBgnhJh2vzOCNKOn41.1_1 2019-12-09T10:20:39.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~yT6a3kE0M8BBcoBgnhJh2vzOCNKOn41.2 2019-12-09T10:20:39.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~yT6a3kE0M8BBcoBgnhJh2vzOCNKOn41.2_1 2019-12-09T10:20:39.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~yT6a3kE0M8BBcoBgnhJh2vzOCNKOn41.3 2019-12-09T10:20:39.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~yT6a3kE0M8BBcoBgnhJh2vzOCNKOn41.3_1 2019-12-09T10:20:39.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~yT6a3kE0M8BBcoBgnhJh2vzOCNKOn41.4 2019-12-09T10:20:39.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~yT6a3kE0M8BBcoBgnhJh2vzOCNKOn41.4_1 2019-12-09T10:20:39.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~yT6a3kE0M8BBcoBgnhJh2vzOCNKOn41.5 2019-12-09T10:20:39.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1233', time=2019-12-09T12:06:13.583091+0000, chain.objs.size()=9 2019-12-09T10:20:39.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~WQpWaBBtAg6fAIEpoVUnOGrhv9-qflx.1 2019-12-09T10:20:39.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.029+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~WQpWaBBtAg6fAIEpoVUnOGrhv9-qflx.1_1 2019-12-09T10:20:39.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.030+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~WQpWaBBtAg6fAIEpoVUnOGrhv9-qflx.2 2019-12-09T10:20:39.350 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.348+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~WQpWaBBtAg6fAIEpoVUnOGrhv9-qflx.2_1 2019-12-09T10:20:39.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.377+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~WQpWaBBtAg6fAIEpoVUnOGrhv9-qflx.3 2019-12-09T10:20:39.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.377+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~WQpWaBBtAg6fAIEpoVUnOGrhv9-qflx.3_1 2019-12-09T10:20:39.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.377+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~WQpWaBBtAg6fAIEpoVUnOGrhv9-qflx.4 2019-12-09T10:20:39.379 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.377+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~WQpWaBBtAg6fAIEpoVUnOGrhv9-qflx.4_1 2019-12-09T10:20:39.380 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.377+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~WQpWaBBtAg6fAIEpoVUnOGrhv9-qflx.5 2019-12-09T10:20:39.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.621+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:39.715 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.713+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:20:39.753 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.751+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:39.769 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.767+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:39.787 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T10:20:39.788 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1088', time=2019-12-09T12:01:32.644121+0000, chain.objs.size()=7 2019-12-09T10:20:39.788 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~wi-yk5U5caMFaLBJ_JavMh34l7IZUdD.1 2019-12-09T10:20:39.788 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~wi-yk5U5caMFaLBJ_JavMh34l7IZUdD.1_1 2019-12-09T10:20:39.788 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~wi-yk5U5caMFaLBJ_JavMh34l7IZUdD.1_2 2019-12-09T10:20:39.788 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~wi-yk5U5caMFaLBJ_JavMh34l7IZUdD.2 2019-12-09T10:20:39.788 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~wi-yk5U5caMFaLBJ_JavMh34l7IZUdD.2_1 2019-12-09T10:20:39.789 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~wi-yk5U5caMFaLBJ_JavMh34l7IZUdD.2_2 2019-12-09T10:20:39.789 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~wi-yk5U5caMFaLBJ_JavMh34l7IZUdD.3 2019-12-09T10:20:39.789 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1137', time=2019-12-09T12:03:10.824675+0000, chain.objs.size()=9 2019-12-09T10:20:39.789 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.71__multipart_key1.2~53rZcGFPknGanv8JWVn84D9hnka50-Q.1 2019-12-09T10:20:39.790 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.71__shadow_key1.2~53rZcGFPknGanv8JWVn84D9hnka50-Q.1_1 2019-12-09T10:20:39.790 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.71__multipart_key1.2~53rZcGFPknGanv8JWVn84D9hnka50-Q.2 2019-12-09T10:20:39.791 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.71__shadow_key1.2~53rZcGFPknGanv8JWVn84D9hnka50-Q.2_1 2019-12-09T10:20:39.791 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:39.785+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.71__multipart_key1.2~53rZcGFPknGanv8JWVn84D9hnka50-Q.3 2019-12-09T10:20:40.054 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.052+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.71__shadow_key1.2~53rZcGFPknGanv8JWVn84D9hnka50-Q.3_1 2019-12-09T10:20:40.055 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.052+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.71__multipart_key1.2~53rZcGFPknGanv8JWVn84D9hnka50-Q.4 2019-12-09T10:20:40.119 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.117+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.71__shadow_key1.2~53rZcGFPknGanv8JWVn84D9hnka50-Q.4_1 2019-12-09T10:20:40.119 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.117+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.71__multipart_key1.2~53rZcGFPknGanv8JWVn84D9hnka50-Q.5 2019-12-09T10:20:40.120 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.117+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1243', time=2019-12-09T12:06:44.177162+0000, chain.objs.size()=7 2019-12-09T10:20:40.120 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.117+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~hjt3HqPdPV6ggCYo85SSBiEJ9HrBQbA.1 2019-12-09T10:20:40.120 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.117+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~hjt3HqPdPV6ggCYo85SSBiEJ9HrBQbA.1_1 2019-12-09T10:20:40.138 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.136+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~hjt3HqPdPV6ggCYo85SSBiEJ9HrBQbA.1_2 2019-12-09T10:20:40.138 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.136+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~hjt3HqPdPV6ggCYo85SSBiEJ9HrBQbA.2 2019-12-09T10:20:40.138 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.136+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~hjt3HqPdPV6ggCYo85SSBiEJ9HrBQbA.2_1 2019-12-09T10:20:40.138 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.136+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~hjt3HqPdPV6ggCYo85SSBiEJ9HrBQbA.2_2 2019-12-09T10:20:40.138 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.136+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~hjt3HqPdPV6ggCYo85SSBiEJ9HrBQbA.3 2019-12-09T10:20:40.341 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.339+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:40.452 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.450+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:20:40.497 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.495+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:40.520 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.518+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:40.530 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.528+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:40.566 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.564+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:20:40.611 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.609+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:40.633 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.631+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:40.649 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.647+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:40.696 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.694+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:20:40.747 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.745+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:40.765 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.762+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:40.772 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.770+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:40.819 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.817+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:20:40.856 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.854+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:40.875 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.873+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:40.883 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.881+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:40.947 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.945+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:20:40.990 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:40.988+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:41.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.009+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:41.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.043+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:20:41.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.043+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1082', time=2019-12-09T12:01:01.618075+0000, chain.objs.size()=9 2019-12-09T10:20:41.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.043+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~yXaO-gZMajZ9-vVKDAg41KPiOdv39SB.1 2019-12-09T10:20:41.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.043+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~yXaO-gZMajZ9-vVKDAg41KPiOdv39SB.1_1 2019-12-09T10:20:41.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.043+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~yXaO-gZMajZ9-vVKDAg41KPiOdv39SB.2 2019-12-09T10:20:41.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.043+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~yXaO-gZMajZ9-vVKDAg41KPiOdv39SB.2_1 2019-12-09T10:20:41.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.043+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~yXaO-gZMajZ9-vVKDAg41KPiOdv39SB.3 2019-12-09T10:20:41.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.044+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~yXaO-gZMajZ9-vVKDAg41KPiOdv39SB.3_1 2019-12-09T10:20:41.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.044+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~yXaO-gZMajZ9-vVKDAg41KPiOdv39SB.4 2019-12-09T10:20:41.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.044+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~yXaO-gZMajZ9-vVKDAg41KPiOdv39SB.4_1 2019-12-09T10:20:41.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.044+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~yXaO-gZMajZ9-vVKDAg41KPiOdv39SB.5 2019-12-09T10:20:41.288 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.286+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:41.367 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.366+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:20:41.427 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.424+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:41.443 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.441+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:41.455 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.453+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:20:41.455 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.453+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1094', time=2019-12-09T12:02:02.480668+0000, chain.objs.size()=7 2019-12-09T10:20:41.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.453+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~6poj3Vlvthym8uHuZZR42TA3PsEi4XF.1 2019-12-09T10:20:41.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.453+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~6poj3Vlvthym8uHuZZR42TA3PsEi4XF.1_1 2019-12-09T10:20:41.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.453+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~6poj3Vlvthym8uHuZZR42TA3PsEi4XF.1_2 2019-12-09T10:20:41.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.453+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~6poj3Vlvthym8uHuZZR42TA3PsEi4XF.2 2019-12-09T10:20:41.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.453+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~6poj3Vlvthym8uHuZZR42TA3PsEi4XF.2_1 2019-12-09T10:20:41.457 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.453+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__shadow_key1.2~6poj3Vlvthym8uHuZZR42TA3PsEi4XF.2_2 2019-12-09T10:20:41.457 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.453+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.67__multipart_key1.2~6poj3Vlvthym8uHuZZR42TA3PsEi4XF.3 2019-12-09T10:20:41.686 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.684+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:41.778 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.776+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:20:41.819 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.817+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:41.837 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.835+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:41.856 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.853+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:20:41.856 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.853+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1045', time=2019-12-09T11:57:59.891712+0000, chain.objs.size()=9 2019-12-09T10:20:41.856 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.853+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__multipart_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.1 2019-12-09T10:20:41.856 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.853+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__shadow_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.1_1 2019-12-09T10:20:41.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.853+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__multipart_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.2 2019-12-09T10:20:41.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.853+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__shadow_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.2_1 2019-12-09T10:20:41.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.853+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__multipart_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.3 2019-12-09T10:20:41.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.853+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__shadow_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.3_1 2019-12-09T10:20:41.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.853+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__multipart_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.4 2019-12-09T10:20:41.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.854+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__shadow_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.4_1 2019-12-09T10:20:41.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.854+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.64__multipart_key1.2~XgOsChMzmOwFGL7uIa5b8e3bpeU0dvV.5 2019-12-09T10:20:41.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.854+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1263', time=2019-12-09T12:07:29.201070+0000, chain.objs.size()=9 2019-12-09T10:20:41.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.854+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.76__multipart_key1.2~cM43SZS2QNjo0VP-b2hAcxC3hUxikr_.1 2019-12-09T10:20:41.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.854+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.76__shadow_key1.2~cM43SZS2QNjo0VP-b2hAcxC3hUxikr_.1_1 2019-12-09T10:20:41.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.854+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.76__multipart_key1.2~cM43SZS2QNjo0VP-b2hAcxC3hUxikr_.2 2019-12-09T10:20:41.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:41.994+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.76__shadow_key1.2~cM43SZS2QNjo0VP-b2hAcxC3hUxikr_.2_1 2019-12-09T10:20:42.032 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.030+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.76__multipart_key1.2~cM43SZS2QNjo0VP-b2hAcxC3hUxikr_.3 2019-12-09T10:20:42.093 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.091+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.76__shadow_key1.2~cM43SZS2QNjo0VP-b2hAcxC3hUxikr_.3_1 2019-12-09T10:20:42.107 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.105+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.76__multipart_key1.2~cM43SZS2QNjo0VP-b2hAcxC3hUxikr_.4 2019-12-09T10:20:42.107 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.105+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.76__shadow_key1.2~cM43SZS2QNjo0VP-b2hAcxC3hUxikr_.4_1 2019-12-09T10:20:42.108 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.105+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.76__multipart_key1.2~cM43SZS2QNjo0VP-b2hAcxC3hUxikr_.5 2019-12-09T10:20:42.277 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.275+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:42.404 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.401+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:20:42.463 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.461+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:42.510 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.508+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:42.525 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.523+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:42.560 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.558+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:20:42.642 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.640+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:42.681 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.680+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:42.712 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:20:42.712 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='2~QrI4wnwHEhk7DgtCPHW5PpmHbRUgwg3', time=2019-12-09T11:55:09.314423+0000, chain.objs.size()=9 2019-12-09T10:20:42.713 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.52__multipart_key1.2~QrI4wnwHEhk7DgtCPHW5PpmHbRUgwg3.1 2019-12-09T10:20:42.713 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.52__shadow_key1.2~QrI4wnwHEhk7DgtCPHW5PpmHbRUgwg3.1_1 2019-12-09T10:20:42.713 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.52__multipart_key1.2~QrI4wnwHEhk7DgtCPHW5PpmHbRUgwg3.2 2019-12-09T10:20:42.713 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.52__shadow_key1.2~QrI4wnwHEhk7DgtCPHW5PpmHbRUgwg3.2_1 2019-12-09T10:20:42.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.52__multipart_key1.2~QrI4wnwHEhk7DgtCPHW5PpmHbRUgwg3.3 2019-12-09T10:20:42.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.52__shadow_key1.2~QrI4wnwHEhk7DgtCPHW5PpmHbRUgwg3.3_1 2019-12-09T10:20:42.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.52__multipart_key1.2~QrI4wnwHEhk7DgtCPHW5PpmHbRUgwg3.4 2019-12-09T10:20:42.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.52__shadow_key1.2~QrI4wnwHEhk7DgtCPHW5PpmHbRUgwg3.4_1 2019-12-09T10:20:42.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.52__multipart_key1.2~QrI4wnwHEhk7DgtCPHW5PpmHbRUgwg3.5 2019-12-09T10:20:42.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1226', time=2019-12-09T12:05:53.808700+0000, chain.objs.size()=9 2019-12-09T10:20:42.715 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~a-GD_-MdqnWRR0ZimYfbwQpCLx18Ktm.1 2019-12-09T10:20:42.715 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~a-GD_-MdqnWRR0ZimYfbwQpCLx18Ktm.1_1 2019-12-09T10:20:42.715 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.710+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~a-GD_-MdqnWRR0ZimYfbwQpCLx18Ktm.2 2019-12-09T10:20:42.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.957+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~a-GD_-MdqnWRR0ZimYfbwQpCLx18Ktm.2_1 2019-12-09T10:20:42.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.957+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~a-GD_-MdqnWRR0ZimYfbwQpCLx18Ktm.3 2019-12-09T10:20:42.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.957+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~a-GD_-MdqnWRR0ZimYfbwQpCLx18Ktm.3_1 2019-12-09T10:20:42.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.957+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~a-GD_-MdqnWRR0ZimYfbwQpCLx18Ktm.4 2019-12-09T10:20:42.998 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.996+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~a-GD_-MdqnWRR0ZimYfbwQpCLx18Ktm.4_1 2019-12-09T10:20:42.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:42.996+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~a-GD_-MdqnWRR0ZimYfbwQpCLx18Ktm.5 2019-12-09T10:20:43.142 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.140+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:43.228 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.226+0000 7f007dabeec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:20:43.272 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.270+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:43.289 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.287+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:43.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:20:43.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1187', time=2019-12-09T12:04:11.246370+0000, chain.objs.size()=9 2019-12-09T10:20:43.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__multipart_file.mpg.2~WFF5dfGZG5v0VOJ1-TSqmOwPcGUQnUf.1 2019-12-09T10:20:43.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__shadow_file.mpg.2~WFF5dfGZG5v0VOJ1-TSqmOwPcGUQnUf.1_1 2019-12-09T10:20:43.308 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__multipart_file.mpg.2~WFF5dfGZG5v0VOJ1-TSqmOwPcGUQnUf.2 2019-12-09T10:20:43.308 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__shadow_file.mpg.2~WFF5dfGZG5v0VOJ1-TSqmOwPcGUQnUf.2_1 2019-12-09T10:20:43.308 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__multipart_file.mpg.2~WFF5dfGZG5v0VOJ1-TSqmOwPcGUQnUf.3 2019-12-09T10:20:43.308 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__shadow_file.mpg.2~WFF5dfGZG5v0VOJ1-TSqmOwPcGUQnUf.3_1 2019-12-09T10:20:43.308 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__multipart_file.mpg.2~WFF5dfGZG5v0VOJ1-TSqmOwPcGUQnUf.4 2019-12-09T10:20:43.308 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__shadow_file.mpg.2~WFF5dfGZG5v0VOJ1-TSqmOwPcGUQnUf.4_1 2019-12-09T10:20:43.308 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.73__multipart_file.mpg.2~WFF5dfGZG5v0VOJ1-TSqmOwPcGUQnUf.5 2019-12-09T10:20:43.309 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 20 garbage collection: RGWGC::process iterating over entry tag='d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4355.1238', time=2019-12-09T12:06:28.918031+0000, chain.objs.size()=7 2019-12-09T10:20:43.309 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~1f41A_VNVFhFc_E2iFgYudSdGu-eHPU.1 2019-12-09T10:20:43.309 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~1f41A_VNVFhFc_E2iFgYudSdGu-eHPU.1_1 2019-12-09T10:20:43.309 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.305+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~1f41A_VNVFhFc_E2iFgYudSdGu-eHPU.1_2 2019-12-09T10:20:43.486 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.484+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~1f41A_VNVFhFc_E2iFgYudSdGu-eHPU.2 2019-12-09T10:20:43.519 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.517+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~1f41A_VNVFhFc_E2iFgYudSdGu-eHPU.2_1 2019-12-09T10:20:43.519 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.517+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__shadow_key1.2~1f41A_VNVFhFc_E2iFgYudSdGu-eHPU.2_2 2019-12-09T10:20:43.520 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.517+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.75__multipart_key1.2~1f41A_VNVFhFc_E2iFgYudSdGu-eHPU.3 2019-12-09T10:20:43.706 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.704+0000 7f007dabeec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:20:43.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.839+0000 7f007dabeec0 20 remove_watcher() i=0 2019-12-09T10:20:43.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.839+0000 7f007dabeec0 2 removed watcher, disabling cache 2019-12-09T10:20:43.880 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.878+0000 7f007dabeec0 20 remove_watcher() i=1 2019-12-09T10:20:43.916 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.914+0000 7f007dabeec0 20 remove_watcher() i=2 2019-12-09T10:20:43.980 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:43.979+0000 7f007dabeec0 20 remove_watcher() i=3 2019-12-09T10:20:44.021 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.019+0000 7f007dabeec0 20 remove_watcher() i=4 2019-12-09T10:20:44.070 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.068+0000 7f007dabeec0 20 remove_watcher() i=5 2019-12-09T10:20:44.115 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.113+0000 7f007dabeec0 20 remove_watcher() i=6 2019-12-09T10:20:44.166 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.164+0000 7f007dabeec0 20 remove_watcher() i=7 2019-12-09T10:20:44.177 INFO:teuthology.orchestra.run.smithi035:> radosgw-admin gc process --include-all 2019-12-09T10:20:44.332 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.330+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.356 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.354+0000 7efe0e2e8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:44.356 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.354+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.366 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.364+0000 7efe0e2e8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:44.366 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.364+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.372 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.370+0000 7efe0e2e8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:44.373 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.370+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.380 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.378+0000 7efe0e2e8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:44.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.629+0000 7efe0e2e8ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T10:20:44.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.629+0000 7efe0e2e8ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T10:20:44.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.629+0000 7efe0e2e8ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:20:44.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.629+0000 7efe0e2e8ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:20:44.631 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.629+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.641 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.639+0000 7efe0e2e8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:44.641 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.639+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.661 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.659+0000 7efe0e2e8ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:20:44.662 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.660+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.669 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.667+0000 7efe0e2e8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:44.669 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.667+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.688 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.686+0000 7efe0e2e8ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:20:44.688 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.686+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.691+0000 7efe0e2e8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:44.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.691+0000 7efe0e2e8ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:20:44.694 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.691+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.702 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.700+0000 7efe0e2e8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:44.702 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.700+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.712 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.710+0000 7efe0e2e8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:44.712 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.710+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.737 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.734+0000 7efe0e2e8ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:20:44.737 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.734+0000 7efe0e2e8ec0 20 zonegroup default 2019-12-09T10:20:44.737 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.734+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.741 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.739+0000 7efe0e2e8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:44.741 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.739+0000 7efe0e2e8ec0 10 Cannot find current period zone using local zone 2019-12-09T10:20:44.742 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.739+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.746 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.744+0000 7efe0e2e8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:44.746 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.744+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.759 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.758+0000 7efe0e2e8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:44.760 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.758+0000 7efe0e2e8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:44.776 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.774+0000 7efe0e2e8ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:20:44.776 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.774+0000 7efe0e2e8ec0 20 zone default found 2019-12-09T10:20:44.776 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.774+0000 7efe0e2e8ec0 20 started sync module instance, tier type = 2019-12-09T10:20:44.776 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:44.774+0000 7efe0e2e8ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T10:20:45.247 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.245+0000 7efe0e2e8ec0 20 add_watcher() i=0 2019-12-09T10:20:45.247 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.245+0000 7efe0e2e8ec0 20 add_watcher() i=1 2019-12-09T10:20:45.248 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.245+0000 7efe0e2e8ec0 20 add_watcher() i=2 2019-12-09T10:20:45.248 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.245+0000 7efe0e2e8ec0 20 add_watcher() i=3 2019-12-09T10:20:45.248 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.245+0000 7efe0e2e8ec0 20 add_watcher() i=4 2019-12-09T10:20:45.248 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.245+0000 7efe0e2e8ec0 20 add_watcher() i=5 2019-12-09T10:20:45.248 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.245+0000 7efe0e2e8ec0 20 add_watcher() i=6 2019-12-09T10:20:45.287 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.285+0000 7efe0e2e8ec0 20 add_watcher() i=7 2019-12-09T10:20:45.287 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.285+0000 7efe0e2e8ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:20:45.287 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.285+0000 7efdcbfdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:20:45.291 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.289+0000 7efdfb512700 0 -- 172.21.15.35:0/327453770 >> v2:172.21.15.22:3300/0 conn(0x55f3f2f551b0 msgr2=0x55f3f2f343c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:20:45.340 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:45.339+0000 7efdfb512700 0 -- 172.21.15.35:0/1779780953 >> [v2:172.21.15.22:6800/8484,v1:172.21.15.22:6801/8484] conn(0x7efd3c0224c0 msgr2=0x7efd3c024990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:20:46.988 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:46.984+0000 7efd3a7fc700 20 reqs_thread_entry: start 2019-12-09T10:20:46.988 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:46.984+0000 7efd33fff700 20 reqs_thread_entry: start 2019-12-09T10:20:46.988 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:46.985+0000 7efe0e2e8ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:20:46.988 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:46.985+0000 7efd327fc700 20 reqs_thread_entry: start 2019-12-09T10:20:46.988 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:46.985+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:20:47.064 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.062+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.087 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.085+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:47.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.099+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.139 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.136+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:20:47.204 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.201+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.220 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.217+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:47.228 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.226+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.273 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.270+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:20:47.313 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.310+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.330 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.327+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:47.339 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.337+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.370 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.367+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:20:47.408 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.406+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.425 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.422+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:47.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.431+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.475 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.472+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:20:47.540 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.537+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.562 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.559+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:47.572 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.569+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.632 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.629+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:20:47.704 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.701+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.728 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.726+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:47.741 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.739+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.779 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:08:24.043 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:47.781 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.778+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:20:47.783 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:08:24.048 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:47.797 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:08:24.020 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:47.800 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:08:24.023 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:47.803 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:08:24.026 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:47.803 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:08:24.027 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:47.828 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:08:24.093 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:47.838 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.835+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.856 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:08:24.079 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:47.861 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.858+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:47.863 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:08:24.086 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:47.874 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.871+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:47.905 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.902+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:20:47.983 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:47.980+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.007 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.004+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:48.020 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.017+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.062 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.059+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:20:48.121 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.118+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.147 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.144+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:48.161 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.158+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.205 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.202+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:20:48.263 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.260+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.293 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.290+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:48.303 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.300+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.336 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.333+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:20:48.410 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.407+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.432 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.429+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:48.445 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.442+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.484 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.481+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:20:48.565 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.562+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.594 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.591+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:48.605 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.602+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.649+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:20:48.706 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.703+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.728 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.725+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:48.741 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.738+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.801 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.798+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:20:48.876 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.873+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.904 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.901+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:48.918 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.915+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:48.973 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:48.970+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:20:49.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.044+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.084 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.082+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:49.103 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.101+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.156 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.154+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:20:49.234 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.232+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.254 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.252+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:49.264 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.263+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.315 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.313+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:20:49.382 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.380+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.415 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.413+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:49.428 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.426+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.484 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.482+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:20:49.526 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.524+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.543 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.542+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:49.553 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.551+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.597 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.595+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:20:49.663 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.661+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.689 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.687+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:49.707 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.705+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.758 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.756+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:20:49.816 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.814+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.836 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.834+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:49.846 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.844+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.887 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.885+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:20:49.960 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.958+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:49.999 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:49.997+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:50.015 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.014+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.073 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.071+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:20:50.136 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.134+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.160 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.158+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:50.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.173+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.230 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.228+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:20:50.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.298+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.324 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.322+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:50.338 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.336+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.503 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.501+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:20:50.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.580+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.608+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:50.621 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.619+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.672 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.670+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:20:50.713 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.711+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.732 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.730+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:50.746 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.740+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.773 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.771+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:20:50.817 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.816+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.834 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.833+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:50.845 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.843+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.885 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.881+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:20:50.963 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.961+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:50.991 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:50.989+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:51.007 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.005+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:51.070 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.068+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:20:51.166 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.164+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:51.203 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.202+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:51.221 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.219+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:51.296 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.294+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:20:51.390 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.388+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:51.419 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.417+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:51.438 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.436+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:51.520 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.518+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:20:51.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.580+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:51.606 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.604+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:51.630 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.629+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:51.668 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.666+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:20:51.706 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.704+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:51.723 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.721+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:51.732 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.731+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:51.772 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.770+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:20:51.824 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.822+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:51.840 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.838+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:20:51.849 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.847+0000 7efe0e2e8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:20:51.960 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.958+0000 7efe0e2e8ec0 20 remove_watcher() i=0 2019-12-09T10:20:51.960 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:51.958+0000 7efe0e2e8ec0 2 removed watcher, disabling cache 2019-12-09T10:20:52.023 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:52.021+0000 7efe0e2e8ec0 20 remove_watcher() i=1 2019-12-09T10:20:52.074 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:52.072+0000 7efe0e2e8ec0 20 remove_watcher() i=2 2019-12-09T10:20:52.148 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:52.145+0000 7efe0e2e8ec0 20 remove_watcher() i=3 2019-12-09T10:20:52.214 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:52.212+0000 7efe0e2e8ec0 20 remove_watcher() i=4 2019-12-09T10:20:52.273 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:52.271+0000 7efe0e2e8ec0 20 remove_watcher() i=5 2019-12-09T10:20:52.345 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:52.343+0000 7efe0e2e8ec0 20 remove_watcher() i=6 2019-12-09T10:20:52.420 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:52.418+0000 7efe0e2e8ec0 20 remove_watcher() i=7 2019-12-09T10:20:52.431 INFO:teuthology.run_tasks:Running task s3tests... 2019-12-09T10:20:52.446 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master', 'slow_backend': True}} 2019-12-09T10:20:52.447 INFO:tasks.s3tests:Downloading s3-tests... 2019-12-09T10:20:52.447 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2019-12-09T10:20:52.447 INFO:teuthology.orchestra.run.smithi035:> git clone -b ceph-master git://git.ceph.com/git/s3-tests.git /home/ubuntu/cephtest/s3-tests 2019-12-09T10:20:52.540 INFO:teuthology.orchestra.run.smithi035.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests'... 2019-12-09T10:20:53.097 INFO:tasks.s3tests:Creating rgw users... 2019-12-09T10:20:53.097 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2019-12-09T10:20:53.097 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name 'Mr. bar.client.0' --access-key EFFIUUJDNZHPZNOAEBSS --secret 9VfQMQ2QISaCodQA690r8cIvY1jlHk0TMW87OxplkCQ6NDGh990iTA== --email bar.client.0+test@test.test --cluster ceph 2019-12-09T10:20:53.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.289+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.313 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.311+0000 7f429bbd1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:53.313 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.311+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.323+0000 7f429bbd1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:53.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.323+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.329 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.328+0000 7f429bbd1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:53.329 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.328+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.340 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.338+0000 7f429bbd1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:53.581 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.575+0000 7f429bbd1ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T10:20:53.581 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.575+0000 7f429bbd1ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T10:20:53.581 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.575+0000 7f429bbd1ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:20:53.581 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.575+0000 7f429bbd1ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:20:53.581 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.575+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.590 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.588+0000 7f429bbd1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:53.590 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.588+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.606 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.604+0000 7f429bbd1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:20:53.606 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.604+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.619 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.617+0000 7f429bbd1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:53.620 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.617+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.639 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.637+0000 7f429bbd1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:20:53.639 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.637+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.645 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.644+0000 7f429bbd1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:53.646 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.644+0000 7f429bbd1ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:20:53.646 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.644+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.650+0000 7f429bbd1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:53.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.650+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.663 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.661+0000 7f429bbd1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:53.663 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.661+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.673 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.671+0000 7f429bbd1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:20:53.673 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.671+0000 7f429bbd1ec0 20 zonegroup default 2019-12-09T10:20:53.673 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.671+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.679 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.677+0000 7f429bbd1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:53.679 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.677+0000 7f429bbd1ec0 10 Cannot find current period zone using local zone 2019-12-09T10:20:53.679 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.677+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.683 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.682+0000 7f429bbd1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:53.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.682+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.696 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.694+0000 7f429bbd1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:53.696 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.694+0000 7f429bbd1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:53.718 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.716+0000 7f429bbd1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:20:53.719 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.716+0000 7f429bbd1ec0 20 zone default found 2019-12-09T10:20:53.719 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.716+0000 7f429bbd1ec0 20 started sync module instance, tier type = 2019-12-09T10:20:53.719 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:53.716+0000 7f429bbd1ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T10:20:54.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.031+0000 7f429bbd1ec0 20 add_watcher() i=0 2019-12-09T10:20:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.031+0000 7f429bbd1ec0 20 add_watcher() i=1 2019-12-09T10:20:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.031+0000 7f429bbd1ec0 20 add_watcher() i=2 2019-12-09T10:20:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.031+0000 7f429bbd1ec0 20 add_watcher() i=3 2019-12-09T10:20:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.031+0000 7f429bbd1ec0 20 add_watcher() i=4 2019-12-09T10:20:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.031+0000 7f429bbd1ec0 20 add_watcher() i=5 2019-12-09T10:20:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.031+0000 7f429bbd1ec0 20 add_watcher() i=6 2019-12-09T10:20:54.064 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.062+0000 7f4288dfb700 0 -- 172.21.15.35:0/4087513755 >> [v2:172.21.15.22:6802/9813,v1:172.21.15.22:6803/9813] conn(0x563f85554f40 msgr2=0x563f85557400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:20:54.064 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.062+0000 7f429bbd1ec0 20 add_watcher() i=7 2019-12-09T10:20:54.064 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.062+0000 7f429bbd1ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:20:54.065 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.062+0000 7f4258fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:20:54.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:54.089+0000 7f4283fff700 0 -- 172.21.15.35:0/521408359 >> v1:172.21.15.22:6789/0 conn(0x563f8556f680 legacy=0x563f85571b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:20:55.528 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.526+0000 7f41caffd700 20 reqs_thread_entry: start 2019-12-09T10:20:55.528 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.526+0000 7f41c8ff9700 20 reqs_thread_entry: start 2019-12-09T10:20:55.528 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.526+0000 7f429bbd1ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:20:55.528 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.526+0000 7f41beffd700 20 reqs_thread_entry: start 2019-12-09T10:20:55.529 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.526+0000 7f429bbd1ec0 20 get_system_obj_state: rctx=0x7ffc59ffbd28 obj=default.rgw.meta:users.uid:bar.client.0 state=0x563f855e84d0 s->prefetch_data=0 2019-12-09T10:20:55.529 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.526+0000 7f429bbd1ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T10:20:55.534 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.532+0000 7f429bbd1ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-12-09T10:20:55.534 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.532+0000 7f429bbd1ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:20:55.535 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.533+0000 7f429bbd1ec0 20 get_system_obj_state: rctx=0x7ffc59ffbd38 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x563f855ea260 s->prefetch_data=0 2019-12-09T10:20:55.535 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.533+0000 7f429bbd1ec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-12-09T10:20:55.539 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.538+0000 7f429bbd1ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-12-09T10:20:55.540 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.538+0000 7f429bbd1ec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T10:20:55.540 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.538+0000 7f429bbd1ec0 20 get_system_obj_state: rctx=0x7ffc59ffbd38 obj=default.rgw.meta:users.keys:EFFIUUJDNZHPZNOAEBSS state=0x563f855ea260 s->prefetch_data=0 2019-12-09T10:20:55.540 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.538+0000 7f429bbd1ec0 10 cache get: name=default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS : miss 2019-12-09T10:20:55.548 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.546+0000 7f429bbd1ec0 10 cache put: name=default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS info.flags=0x0 2019-12-09T10:20:55.548 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.546+0000 7f429bbd1ec0 10 adding default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS to cache LRU end 2019-12-09T10:20:55.548 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.546+0000 7f429bbd1ec0 20 get_system_obj_state: rctx=0x7ffc59ffb948 obj=default.rgw.meta:users.keys:EFFIUUJDNZHPZNOAEBSS state=0x563f855ea260 s->prefetch_data=0 2019-12-09T10:20:55.548 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.546+0000 7f429bbd1ec0 10 cache get: name=default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS : type miss (requested=0x6, cached=0x0) 2019-12-09T10:20:55.555 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.553+0000 7f429bbd1ec0 10 cache put: name=default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS info.flags=0x0 2019-12-09T10:20:55.555 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.553+0000 7f429bbd1ec0 10 moving default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS to cache LRU end 2019-12-09T10:20:55.555 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.553+0000 7f429bbd1ec0 20 get_system_obj_state: rctx=0x7ffc59ffbac8 obj=default.rgw.meta:users.keys:EFFIUUJDNZHPZNOAEBSS state=0x563f855ea260 s->prefetch_data=0 2019-12-09T10:20:55.556 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.553+0000 7f429bbd1ec0 10 cache get: name=default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS : type miss (requested=0x6, cached=0x0) 2019-12-09T10:20:55.571 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.560+0000 7f429bbd1ec0 10 cache put: name=default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS info.flags=0x0 2019-12-09T10:20:55.572 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.560+0000 7f429bbd1ec0 10 moving default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS to cache LRU end 2019-12-09T10:20:55.605 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.604+0000 7f429bbd1ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T10:20:55.606 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.604+0000 7f429bbd1ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:20:55.606 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.604+0000 7f429bbd1ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=617 2019-12-09T10:20:55.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.621+0000 7f4271ffb700 10 RGWWatcher::handle_notify() notify_id 575525617692 cookie 94830819859136 notifier 4637 bl.length()=617 2019-12-09T10:20:55.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.621+0000 7f4271ffb700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T10:20:55.623 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.621+0000 7f4271ffb700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:20:55.698 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.696+0000 7f429bbd1ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T10:20:55.699 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.696+0000 7f429bbd1ec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T10:20:55.699 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.696+0000 7f429bbd1ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-12-09T10:20:55.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.706+0000 7f4271ffb700 10 RGWWatcher::handle_notify() notify_id 575525617693 cookie 94830819859136 notifier 4637 bl.length()=194 2019-12-09T10:20:55.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.706+0000 7f4271ffb700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T10:20:55.708 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:55.706+0000 7f4271ffb700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T10:20:56.145 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.143+0000 7f429bbd1ec0 10 cache put: name=default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS info.flags=0x7 2019-12-09T10:20:56.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.143+0000 7f429bbd1ec0 10 moving default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS to cache LRU end 2019-12-09T10:20:56.146 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.143+0000 7f429bbd1ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=186 2019-12-09T10:20:56.171 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.169+0000 7f4271ffb700 10 RGWWatcher::handle_notify() notify_id 575525617708 cookie 94830819880672 notifier 4637 bl.length()=186 2019-12-09T10:20:56.171 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.169+0000 7f4271ffb700 10 cache put: name=default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS info.flags=0x7 2019-12-09T10:20:56.171 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.169+0000 7f4271ffb700 10 moving default.rgw.meta+users.keys+EFFIUUJDNZHPZNOAEBSS to cache LRU end 2019-12-09T10:20:56.593 INFO:teuthology.orchestra.run.smithi035.stdout:{ 2019-12-09T10:20:56.594 INFO:teuthology.orchestra.run.smithi035.stdout: "user_id": "bar.client.0", 2019-12-09T10:20:56.594 INFO:teuthology.orchestra.run.smithi035.stdout: "display_name": "Mr. bar.client.0", 2019-12-09T10:20:56.594 INFO:teuthology.orchestra.run.smithi035.stdout: "email": "bar.client.0+test@test.test", 2019-12-09T10:20:56.594 INFO:teuthology.orchestra.run.smithi035.stdout: "suspended": 0, 2019-12-09T10:20:56.594 INFO:teuthology.orchestra.run.smithi035.stdout: "max_buckets": 1000, 2019-12-09T10:20:56.595 INFO:teuthology.orchestra.run.smithi035.stdout: "subusers": [], 2019-12-09T10:20:56.595 INFO:teuthology.orchestra.run.smithi035.stdout: "keys": [ 2019-12-09T10:20:56.595 INFO:teuthology.orchestra.run.smithi035.stdout: { 2019-12-09T10:20:56.595 INFO:teuthology.orchestra.run.smithi035.stdout: "user": "bar.client.0", 2019-12-09T10:20:56.595 INFO:teuthology.orchestra.run.smithi035.stdout: "access_key": "EFFIUUJDNZHPZNOAEBSS", 2019-12-09T10:20:56.595 INFO:teuthology.orchestra.run.smithi035.stdout: "secret_key": "9VfQMQ2QISaCodQA690r8cIvY1jlHk0TMW87OxplkCQ6NDGh990iTA==" 2019-12-09T10:20:56.595 INFO:teuthology.orchestra.run.smithi035.stdout: } 2019-12-09T10:20:56.595 INFO:teuthology.orchestra.run.smithi035.stdout: ], 2019-12-09T10:20:56.596 INFO:teuthology.orchestra.run.smithi035.stdout: "swift_keys": [], 2019-12-09T10:20:56.596 INFO:teuthology.orchestra.run.smithi035.stdout: "caps": [], 2019-12-09T10:20:56.596 INFO:teuthology.orchestra.run.smithi035.stdout: "op_mask": "read, write, delete", 2019-12-09T10:20:56.596 INFO:teuthology.orchestra.run.smithi035.stdout: "default_placement": "", 2019-12-09T10:20:56.596 INFO:teuthology.orchestra.run.smithi035.stdout: "default_storage_class": "", 2019-12-09T10:20:56.596 INFO:teuthology.orchestra.run.smithi035.stdout: "placement_tags": [], 2019-12-09T10:20:56.596 INFO:teuthology.orchestra.run.smithi035.stdout: "bucket_quota": { 2019-12-09T10:20:56.597 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T10:20:56.597 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T10:20:56.597 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T10:20:56.597 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T10:20:56.597 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T10:20:56.597 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T10:20:56.597 INFO:teuthology.orchestra.run.smithi035.stdout: "user_quota": { 2019-12-09T10:20:56.598 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T10:20:56.598 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T10:20:56.598 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T10:20:56.598 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T10:20:56.598 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T10:20:56.598 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T10:20:56.598 INFO:teuthology.orchestra.run.smithi035.stdout: "temp_url_keys": [], 2019-12-09T10:20:56.599 INFO:teuthology.orchestra.run.smithi035.stdout: "type": "rgw", 2019-12-09T10:20:56.599 INFO:teuthology.orchestra.run.smithi035.stdout: "mfa_ids": [] 2019-12-09T10:20:56.599 INFO:teuthology.orchestra.run.smithi035.stdout:} 2019-12-09T10:20:56.599 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:20:56.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.654+0000 7f429bbd1ec0 20 remove_watcher() i=0 2019-12-09T10:20:56.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.654+0000 7f429bbd1ec0 2 removed watcher, disabling cache 2019-12-09T10:20:56.717 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.716+0000 7f429bbd1ec0 20 remove_watcher() i=1 2019-12-09T10:20:56.769 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.767+0000 7f429bbd1ec0 20 remove_watcher() i=2 2019-12-09T10:20:56.831 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.829+0000 7f429bbd1ec0 20 remove_watcher() i=3 2019-12-09T10:20:56.880 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.878+0000 7f429bbd1ec0 20 remove_watcher() i=4 2019-12-09T10:20:56.933 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.932+0000 7f429bbd1ec0 20 remove_watcher() i=5 2019-12-09T10:20:56.983 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:56.981+0000 7f429bbd1ec0 20 remove_watcher() i=6 2019-12-09T10:20:57.060 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.058+0000 7f429bbd1ec0 20 remove_watcher() i=7 2019-12-09T10:20:57.068 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 9340232206 --totp-seed Q4KMI6G5O4B6I3ZNWK2YDODTOFPJAQOOGMPDX2KVV3TQTJEF3BUABNQ26D24LJW2 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T10:20:57.196 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.195+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.222+0000 7f3f8cbaaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:57.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.222+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.235 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.233+0000 7f3f8cbaaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:57.235 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.233+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.240 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.238+0000 7f3f8cbaaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:57.240 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.238+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.246 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.245+0000 7f3f8cbaaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:57.432 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.430+0000 7f3f8cbaaec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T10:20:57.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.430+0000 7f3f8cbaaec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T10:20:57.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.430+0000 7f3f8cbaaec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:20:57.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.430+0000 7f3f8cbaaec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:20:57.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.430+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.443 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.442+0000 7f3f8cbaaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:57.443 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.442+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.463+0000 7f3f8cbaaec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:20:57.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.463+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.481 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.479+0000 7f3f8cbaaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:57.481 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.479+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.496 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.494+0000 7f3f8cbaaec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:20:57.496 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.494+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.504 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.503+0000 7f3f8cbaaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:57.505 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.503+0000 7f3f8cbaaec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:20:57.505 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.503+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.511 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.509+0000 7f3f8cbaaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:57.511 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.509+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.523 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.521+0000 7f3f8cbaaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:57.523 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.521+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.546 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.544+0000 7f3f8cbaaec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:20:57.546 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.544+0000 7f3f8cbaaec0 20 zonegroup default 2019-12-09T10:20:57.547 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.544+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.552 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.551+0000 7f3f8cbaaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:57.553 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.551+0000 7f3f8cbaaec0 10 Cannot find current period zone using local zone 2019-12-09T10:20:57.553 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.551+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.557+0000 7f3f8cbaaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:20:57.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.557+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.569 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.567+0000 7f3f8cbaaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:20:57.569 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.568+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:57.588 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.585+0000 7f3f8cbaaec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:20:57.588 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.586+0000 7f3f8cbaaec0 20 zone default found 2019-12-09T10:20:57.588 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.586+0000 7f3f8cbaaec0 20 started sync module instance, tier type = 2019-12-09T10:20:57.589 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.586+0000 7f3f8cbaaec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T10:20:58.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.999+0000 7f3f8cbaaec0 20 add_watcher() i=0 2019-12-09T10:20:58.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.999+0000 7f3f8cbaaec0 20 add_watcher() i=1 2019-12-09T10:20:58.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.999+0000 7f3f8cbaaec0 20 add_watcher() i=2 2019-12-09T10:20:58.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.999+0000 7f3f8cbaaec0 20 add_watcher() i=3 2019-12-09T10:20:58.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.999+0000 7f3f8cbaaec0 20 add_watcher() i=4 2019-12-09T10:20:58.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:57.999+0000 7f3f8cbaaec0 20 add_watcher() i=5 2019-12-09T10:20:58.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:58.009+0000 7f3f8cbaaec0 20 add_watcher() i=6 2019-12-09T10:20:58.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:58.067+0000 7f3f8cbaaec0 20 add_watcher() i=7 2019-12-09T10:20:58.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:58.067+0000 7f3f8cbaaec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:20:58.070 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:58.067+0000 7f3f497ea700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:20:58.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:58.084+0000 7f3f79dd4700 0 -- 172.21.15.35:0/140465956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca8ac0d220 msgr2=0x55ca8ac0f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:20:58.131 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:58.129+0000 7f3f715d3700 0 -- 172.21.15.35:0/2973749765 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x55ca8ac17270 msgr2=0x55ca8ac19780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:20:59.656 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.654+0000 7f3ebb7fe700 20 reqs_thread_entry: start 2019-12-09T10:20:59.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.655+0000 7f3eb97fa700 20 reqs_thread_entry: start 2019-12-09T10:20:59.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.655+0000 7f3f8cbaaec0 20 init_complete bucket index max shards: 0 2019-12-09T10:20:59.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.656+0000 7f3ea77fe700 20 reqs_thread_entry: start 2019-12-09T10:20:59.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.656+0000 7f3f8cbaaec0 20 get_system_obj_state: rctx=0x7ffcae237108 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55ca8ac8c1c0 s->prefetch_data=0 2019-12-09T10:20:59.658 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.656+0000 7f3f8cbaaec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T10:20:59.690 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.688+0000 7f3f8cbaaec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-12-09T10:20:59.690 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.688+0000 7f3f8cbaaec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:20:59.691 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.688+0000 7f3f8cbaaec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:20:59.691 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.688+0000 7f3f8cbaaec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:20:59.691 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.688+0000 7f3f8cbaaec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T10:20:59.691 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.688+0000 7f3f8cbaaec0 20 rados->read ofs=0 len=0 2019-12-09T10:20:59.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.712+0000 7f3f8cbaaec0 20 rados_obj.operate() r=0 bl.length=432 2019-12-09T10:20:59.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.712+0000 7f3f8cbaaec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-12-09T10:20:59.714 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:20:59.712+0000 7f3f8cbaaec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:21:02.929 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:02.927+0000 7f3f8cbaaec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T10:21:02.930 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:02.927+0000 7f3f8cbaaec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:21:02.930 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:02.927+0000 7f3f8cbaaec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=631 2019-12-09T10:21:02.945 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:02.943+0000 7f3f6affd700 10 RGWWatcher::handle_notify() notify_id 584115552286 cookie 94328399542688 notifier 4945 bl.length()=631 2019-12-09T10:21:02.945 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:02.943+0000 7f3f6affd700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T10:21:02.946 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:02.943+0000 7f3f6affd700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:21:03.213 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:03.210+0000 7f3f8cbaaec0 20 remove_watcher() i=0 2019-12-09T10:21:03.213 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:03.210+0000 7f3f8cbaaec0 2 removed watcher, disabling cache 2019-12-09T10:21:03.271 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:03.269+0000 7f3f8cbaaec0 20 remove_watcher() i=1 2019-12-09T10:21:03.969 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:03.966+0000 7f3f8cbaaec0 20 remove_watcher() i=2 2019-12-09T10:21:04.027 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.025+0000 7f3f8cbaaec0 20 remove_watcher() i=3 2019-12-09T10:21:04.104 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.102+0000 7f3f8cbaaec0 20 remove_watcher() i=4 2019-12-09T10:21:04.170 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.168+0000 7f3f8cbaaec0 20 remove_watcher() i=5 2019-12-09T10:21:04.234 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.232+0000 7f3f8cbaaec0 20 remove_watcher() i=6 2019-12-09T10:21:04.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.295+0000 7f3f8cbaaec0 20 remove_watcher() i=7 2019-12-09T10:21:04.312 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2019-12-09T10:21:04.312 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name 'Mr. foo.client.0' --access-key AHJBSMMNUQWHBYXCCQJU --secret +FXn6cOwhuIDuNsFCh5eldqIuNUZnX9g1lPFJbXasSjOMYq9YsyfIw== --email foo.client.0+test@test.test --cluster ceph 2019-12-09T10:21:04.510 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.508+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.545 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.543+0000 7fdcad313ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:04.545 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.543+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.558 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.556+0000 7fdcad313ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:04.558 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.556+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.565 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.563+0000 7fdcad313ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:04.565 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.563+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.577 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.575+0000 7fdcad313ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:04.797 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.795+0000 7fdcad313ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T10:21:04.797 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.795+0000 7fdcad313ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T10:21:04.797 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.795+0000 7fdcad313ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:21:04.797 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.795+0000 7fdcad313ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:21:04.798 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.795+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.807 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.806+0000 7fdcad313ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:04.808 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.806+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.829 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.827+0000 7fdcad313ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:21:04.829 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.827+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.839 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.837+0000 7fdcad313ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:04.839 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.837+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.849+0000 7fdcad313ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:21:04.851 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.849+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.856+0000 7fdcad313ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:04.858 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.856+0000 7fdcad313ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:21:04.859 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.856+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.867 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.865+0000 7fdcad313ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:04.867 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.865+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.877+0000 7fdcad313ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:04.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.877+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.902 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.898+0000 7fdcad313ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:21:04.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.898+0000 7fdcad313ec0 20 zonegroup default 2019-12-09T10:21:04.903 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.898+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.906+0000 7fdcad313ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:04.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.906+0000 7fdcad313ec0 10 Cannot find current period zone using local zone 2019-12-09T10:21:04.908 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.906+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.915 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.913+0000 7fdcad313ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:04.915 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.913+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.925+0000 7fdcad313ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:04.927 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.925+0000 7fdcad313ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:04.945 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.943+0000 7fdcad313ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:21:04.946 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.944+0000 7fdcad313ec0 20 zone default found 2019-12-09T10:21:04.946 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.944+0000 7fdcad313ec0 20 started sync module instance, tier type = 2019-12-09T10:21:04.946 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:04.944+0000 7fdcad313ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T10:21:05.346 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.344+0000 7fdcad313ec0 20 add_watcher() i=0 2019-12-09T10:21:05.346 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.344+0000 7fdcad313ec0 20 add_watcher() i=1 2019-12-09T10:21:05.346 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.344+0000 7fdcad313ec0 20 add_watcher() i=2 2019-12-09T10:21:05.347 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.344+0000 7fdcad313ec0 20 add_watcher() i=3 2019-12-09T10:21:05.347 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.344+0000 7fdcad313ec0 20 add_watcher() i=4 2019-12-09T10:21:05.347 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.344+0000 7fdcad313ec0 20 add_watcher() i=5 2019-12-09T10:21:05.347 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.344+0000 7fdcad313ec0 20 add_watcher() i=6 2019-12-09T10:21:05.384 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.382+0000 7fdcad313ec0 20 add_watcher() i=7 2019-12-09T10:21:05.384 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.382+0000 7fdcad313ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:21:05.384 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.382+0000 7fdc6afdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:21:05.399 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.398+0000 7fdc9a53d700 0 -- 172.21.15.35:0/1039857758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556366ece4f0 msgr2=0x556366ed4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:21:05.438 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:05.436+0000 7fdc9ad3e700 0 -- 172.21.15.35:0/1009632984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556366ed44b0 msgr2=0x556366ed48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:21:07.081 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.079+0000 7fdbd97fa700 20 reqs_thread_entry: start 2019-12-09T10:21:07.082 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.079+0000 7fdbd2ffd700 20 reqs_thread_entry: start 2019-12-09T10:21:07.082 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.080+0000 7fdcad313ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:21:07.082 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.080+0000 7fdbd17fa700 20 reqs_thread_entry: start 2019-12-09T10:21:07.082 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.081+0000 7fdcad313ec0 20 get_system_obj_state: rctx=0x7ffeec824d88 obj=default.rgw.meta:users.uid:foo.client.0 state=0x556366f4c810 s->prefetch_data=0 2019-12-09T10:21:07.083 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.081+0000 7fdcad313ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T10:21:07.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.095+0000 7fdcad313ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-12-09T10:21:07.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.095+0000 7fdcad313ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:21:07.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.095+0000 7fdcad313ec0 20 get_system_obj_state: rctx=0x7ffeec824d98 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x556366f4e5a0 s->prefetch_data=0 2019-12-09T10:21:07.098 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.095+0000 7fdcad313ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-12-09T10:21:07.106 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.104+0000 7fdcad313ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-12-09T10:21:07.106 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.104+0000 7fdcad313ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T10:21:07.106 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.104+0000 7fdcad313ec0 20 get_system_obj_state: rctx=0x7ffeec824d98 obj=default.rgw.meta:users.keys:AHJBSMMNUQWHBYXCCQJU state=0x556366f4e5a0 s->prefetch_data=0 2019-12-09T10:21:07.107 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.104+0000 7fdcad313ec0 10 cache get: name=default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU : miss 2019-12-09T10:21:07.119 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.117+0000 7fdcad313ec0 10 cache put: name=default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU info.flags=0x0 2019-12-09T10:21:07.119 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.117+0000 7fdcad313ec0 10 adding default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU to cache LRU end 2019-12-09T10:21:07.119 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.117+0000 7fdcad313ec0 20 get_system_obj_state: rctx=0x7ffeec8249a8 obj=default.rgw.meta:users.keys:AHJBSMMNUQWHBYXCCQJU state=0x556366f4e5a0 s->prefetch_data=0 2019-12-09T10:21:07.119 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.117+0000 7fdcad313ec0 10 cache get: name=default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU : type miss (requested=0x6, cached=0x0) 2019-12-09T10:21:07.134 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.132+0000 7fdcad313ec0 10 cache put: name=default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU info.flags=0x0 2019-12-09T10:21:07.134 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.132+0000 7fdcad313ec0 10 moving default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU to cache LRU end 2019-12-09T10:21:07.135 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.132+0000 7fdcad313ec0 20 get_system_obj_state: rctx=0x7ffeec824b28 obj=default.rgw.meta:users.keys:AHJBSMMNUQWHBYXCCQJU state=0x556366f4e5a0 s->prefetch_data=0 2019-12-09T10:21:07.135 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.132+0000 7fdcad313ec0 10 cache get: name=default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU : type miss (requested=0x6, cached=0x0) 2019-12-09T10:21:07.142 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.140+0000 7fdcad313ec0 10 cache put: name=default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU info.flags=0x0 2019-12-09T10:21:07.142 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.140+0000 7fdcad313ec0 10 moving default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU to cache LRU end 2019-12-09T10:21:07.213 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.211+0000 7fdcad313ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T10:21:07.214 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.211+0000 7fdcad313ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:21:07.214 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.211+0000 7fdcad313ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=617 2019-12-09T10:21:07.228 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.226+0000 7fdc7b7fe700 10 RGWWatcher::handle_notify() notify_id 588410519577 cookie 93885416821376 notifier 4872 bl.length()=617 2019-12-09T10:21:07.228 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.226+0000 7fdc7b7fe700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T10:21:07.228 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.226+0000 7fdc7b7fe700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:21:07.413 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.411+0000 7fdcad313ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T10:21:07.413 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.411+0000 7fdcad313ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T10:21:07.413 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.411+0000 7fdcad313ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-12-09T10:21:07.437 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.435+0000 7fdc7b7fe700 10 RGWWatcher::handle_notify() notify_id 588410519602 cookie 93885416850976 notifier 4872 bl.length()=194 2019-12-09T10:21:07.437 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.435+0000 7fdc7b7fe700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T10:21:07.437 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.435+0000 7fdc7b7fe700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T10:21:07.630 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.628+0000 7fdcad313ec0 10 cache put: name=default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU info.flags=0x7 2019-12-09T10:21:07.630 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.628+0000 7fdcad313ec0 10 moving default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU to cache LRU end 2019-12-09T10:21:07.630 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.628+0000 7fdcad313ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=186 2019-12-09T10:21:07.642 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.640+0000 7fdc7b7fe700 10 RGWWatcher::handle_notify() notify_id 588410519629 cookie 93885416847392 notifier 4872 bl.length()=186 2019-12-09T10:21:07.642 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.640+0000 7fdc7b7fe700 10 cache put: name=default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU info.flags=0x7 2019-12-09T10:21:07.642 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:07.640+0000 7fdc7b7fe700 10 moving default.rgw.meta+users.keys+AHJBSMMNUQWHBYXCCQJU to cache LRU end 2019-12-09T10:21:07.966 INFO:teuthology.orchestra.run.smithi035.stdout:{ 2019-12-09T10:21:07.966 INFO:teuthology.orchestra.run.smithi035.stdout: "user_id": "foo.client.0", 2019-12-09T10:21:07.967 INFO:teuthology.orchestra.run.smithi035.stdout: "display_name": "Mr. foo.client.0", 2019-12-09T10:21:07.967 INFO:teuthology.orchestra.run.smithi035.stdout: "email": "foo.client.0+test@test.test", 2019-12-09T10:21:07.967 INFO:teuthology.orchestra.run.smithi035.stdout: "suspended": 0, 2019-12-09T10:21:07.967 INFO:teuthology.orchestra.run.smithi035.stdout: "max_buckets": 1000, 2019-12-09T10:21:07.967 INFO:teuthology.orchestra.run.smithi035.stdout: "subusers": [], 2019-12-09T10:21:07.967 INFO:teuthology.orchestra.run.smithi035.stdout: "keys": [ 2019-12-09T10:21:07.968 INFO:teuthology.orchestra.run.smithi035.stdout: { 2019-12-09T10:21:07.968 INFO:teuthology.orchestra.run.smithi035.stdout: "user": "foo.client.0", 2019-12-09T10:21:07.968 INFO:teuthology.orchestra.run.smithi035.stdout: "access_key": "AHJBSMMNUQWHBYXCCQJU", 2019-12-09T10:21:07.969 INFO:teuthology.orchestra.run.smithi035.stdout: "secret_key": "+FXn6cOwhuIDuNsFCh5eldqIuNUZnX9g1lPFJbXasSjOMYq9YsyfIw==" 2019-12-09T10:21:07.969 INFO:teuthology.orchestra.run.smithi035.stdout: } 2019-12-09T10:21:07.969 INFO:teuthology.orchestra.run.smithi035.stdout: ], 2019-12-09T10:21:07.969 INFO:teuthology.orchestra.run.smithi035.stdout: "swift_keys": [], 2019-12-09T10:21:07.969 INFO:teuthology.orchestra.run.smithi035.stdout: "caps": [], 2019-12-09T10:21:07.970 INFO:teuthology.orchestra.run.smithi035.stdout: "op_mask": "read, write, delete", 2019-12-09T10:21:07.970 INFO:teuthology.orchestra.run.smithi035.stdout: "default_placement": "", 2019-12-09T10:21:07.970 INFO:teuthology.orchestra.run.smithi035.stdout: "default_storage_class": "", 2019-12-09T10:21:07.970 INFO:teuthology.orchestra.run.smithi035.stdout: "placement_tags": [], 2019-12-09T10:21:07.971 INFO:teuthology.orchestra.run.smithi035.stdout: "bucket_quota": { 2019-12-09T10:21:07.971 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T10:21:07.971 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T10:21:07.971 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T10:21:07.971 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T10:21:07.972 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T10:21:07.972 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T10:21:07.975 INFO:teuthology.orchestra.run.smithi035.stdout: "user_quota": { 2019-12-09T10:21:07.975 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T10:21:07.975 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T10:21:07.976 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T10:21:07.976 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T10:21:07.976 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T10:21:07.976 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T10:21:07.977 INFO:teuthology.orchestra.run.smithi035.stdout: "temp_url_keys": [], 2019-12-09T10:21:07.977 INFO:teuthology.orchestra.run.smithi035.stdout: "type": "rgw", 2019-12-09T10:21:07.977 INFO:teuthology.orchestra.run.smithi035.stdout: "mfa_ids": [] 2019-12-09T10:21:07.978 INFO:teuthology.orchestra.run.smithi035.stdout:} 2019-12-09T10:21:07.978 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:08.042 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.040+0000 7fdcad313ec0 20 remove_watcher() i=0 2019-12-09T10:21:08.042 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.040+0000 7fdcad313ec0 2 removed watcher, disabling cache 2019-12-09T10:21:08.109 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.108+0000 7fdcad313ec0 20 remove_watcher() i=1 2019-12-09T10:21:08.181 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.175+0000 7fdcad313ec0 20 remove_watcher() i=2 2019-12-09T10:21:08.243 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.241+0000 7fdcad313ec0 20 remove_watcher() i=3 2019-12-09T10:21:08.307 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.305+0000 7fdcad313ec0 20 remove_watcher() i=4 2019-12-09T10:21:08.359 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.357+0000 7fdcad313ec0 20 remove_watcher() i=5 2019-12-09T10:21:08.412 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.410+0000 7fdcad313ec0 20 remove_watcher() i=6 2019-12-09T10:21:08.497 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.495+0000 7fdcad313ec0 20 remove_watcher() i=7 2019-12-09T10:21:08.510 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 8242603595 --totp-seed VLTMUEWLX6BV6KVMX7BORY6K323S4YUZ2BYRLVTHPCT5YEE6X6V2RNKTZEMPVKT4 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T10:21:08.704 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.701+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:08.748 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.746+0000 7f4e455f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:08.748 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.746+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:08.758 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.757+0000 7f4e455f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:08.759 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.757+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:08.764 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.762+0000 7f4e455f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:08.764 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.762+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:08.773 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.771+0000 7f4e455f1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:08.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.994+0000 7f4e455f1ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T10:21:08.997 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.994+0000 7f4e455f1ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T10:21:08.997 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.994+0000 7f4e455f1ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:21:08.997 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.994+0000 7f4e455f1ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:21:08.997 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:08.994+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.006 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.004+0000 7f4e455f1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:09.006 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.004+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.027 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.025+0000 7f4e455f1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:21:09.027 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.025+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.038 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.036+0000 7f4e455f1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:09.038 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.036+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.055+0000 7f4e455f1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:21:09.058 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.056+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.063 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.061+0000 7f4e455f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:09.063 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.061+0000 7f4e455f1ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:21:09.063 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.061+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.067 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.065+0000 7f4e455f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:09.067 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.065+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.074+0000 7f4e455f1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:09.076 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.074+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.098 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.095+0000 7f4e455f1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:21:09.098 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.095+0000 7f4e455f1ec0 20 zonegroup default 2019-12-09T10:21:09.098 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.095+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.101+0000 7f4e455f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:09.102 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.101+0000 7f4e455f1ec0 10 Cannot find current period zone using local zone 2019-12-09T10:21:09.103 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.101+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.110 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.108+0000 7f4e455f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:09.110 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.108+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.120 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.119+0000 7f4e455f1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:09.121 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.119+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:09.140 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.138+0000 7f4e455f1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:21:09.140 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.138+0000 7f4e455f1ec0 20 zone default found 2019-12-09T10:21:09.141 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.138+0000 7f4e455f1ec0 20 started sync module instance, tier type = 2019-12-09T10:21:09.141 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.138+0000 7f4e455f1ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T10:21:09.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.556+0000 7f4e455f1ec0 20 add_watcher() i=0 2019-12-09T10:21:09.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.556+0000 7f4e455f1ec0 20 add_watcher() i=1 2019-12-09T10:21:09.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.557+0000 7f4e455f1ec0 20 add_watcher() i=2 2019-12-09T10:21:09.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.557+0000 7f4e455f1ec0 20 add_watcher() i=3 2019-12-09T10:21:09.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.557+0000 7f4e455f1ec0 20 add_watcher() i=4 2019-12-09T10:21:09.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.557+0000 7f4e455f1ec0 20 add_watcher() i=5 2019-12-09T10:21:09.560 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.557+0000 7f4e455f1ec0 20 add_watcher() i=6 2019-12-09T10:21:09.593 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.591+0000 7f4e455f1ec0 20 add_watcher() i=7 2019-12-09T10:21:09.594 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.591+0000 7f4e455f1ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:21:09.594 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.591+0000 7f4e027ec700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:21:09.620 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.614+0000 7f4e3281b700 0 -- >> v1:172.21.15.22:6789/0 conn(0x561420954d00 legacy=0x561420957120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:21:09.760 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:09.758+0000 7f4e3281b700 0 -- 172.21.15.35:0/1648350640 >> [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] conn(0x56142095cc30 msgr2=0x56142095d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:21:11.454 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.452+0000 7f4d78ff9700 20 reqs_thread_entry: start 2019-12-09T10:21:11.454 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.452+0000 7f4d727fc700 20 reqs_thread_entry: start 2019-12-09T10:21:11.455 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.452+0000 7f4e455f1ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:21:11.455 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.453+0000 7f4d70ff9700 20 reqs_thread_entry: start 2019-12-09T10:21:11.455 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.453+0000 7f4e455f1ec0 20 get_system_obj_state: rctx=0x7ffec3a9d9b8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x5614209d1630 s->prefetch_data=0 2019-12-09T10:21:11.456 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.453+0000 7f4e455f1ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T10:21:11.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.475+0000 7f4e455f1ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-12-09T10:21:11.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.475+0000 7f4e455f1ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:21:11.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.475+0000 7f4e455f1ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:21:11.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.475+0000 7f4e455f1ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:21:11.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.475+0000 7f4e455f1ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T10:21:11.478 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.475+0000 7f4e455f1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:11.504 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.502+0000 7f4e455f1ec0 20 rados_obj.operate() r=0 bl.length=432 2019-12-09T10:21:11.504 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.502+0000 7f4e455f1ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-12-09T10:21:11.505 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.502+0000 7f4e455f1ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:21:11.674 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.672+0000 7f4e455f1ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T10:21:11.675 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.672+0000 7f4e455f1ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:21:11.675 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.672+0000 7f4e455f1ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=631 2019-12-09T10:21:11.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.682+0000 7f4e23fff700 10 RGWWatcher::handle_notify() notify_id 588410519578 cookie 94644445857504 notifier 4963 bl.length()=631 2019-12-09T10:21:11.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.682+0000 7f4e23fff700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T10:21:11.684 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:11.682+0000 7f4e23fff700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:21:12.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.173+0000 7f4e455f1ec0 20 remove_watcher() i=0 2019-12-09T10:21:12.175 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.173+0000 7f4e455f1ec0 2 removed watcher, disabling cache 2019-12-09T10:21:12.220 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.218+0000 7f4e455f1ec0 20 remove_watcher() i=1 2019-12-09T10:21:12.274 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.272+0000 7f4e455f1ec0 20 remove_watcher() i=2 2019-12-09T10:21:12.340 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.338+0000 7f4e455f1ec0 20 remove_watcher() i=3 2019-12-09T10:21:12.392 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.390+0000 7f4e455f1ec0 20 remove_watcher() i=4 2019-12-09T10:21:12.439 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.437+0000 7f4e455f1ec0 20 remove_watcher() i=5 2019-12-09T10:21:12.492 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.489+0000 7f4e455f1ec0 20 remove_watcher() i=6 2019-12-09T10:21:12.541 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.539+0000 7f4e455f1ec0 20 remove_watcher() i=7 2019-12-09T10:21:12.556 DEBUG:tasks.s3tests:Creating user testx$tenanteduser.client.0 on client.0 2019-12-09T10:21:12.556 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name 'Mr. testx$tenanteduser.client.0' --access-key YTABYFXHGYOWVVUVVLWN --secret PkCoQQpWjZPyn8l2m8IQAdZ+NJfZn7/uJWMkGTMxMGBi9JRHsJVscw== --email 'testx$tenanteduser.client.0+test@test.test' --cluster ceph 2019-12-09T10:21:12.699 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.697+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:12.726 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.724+0000 7f06ca4fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:12.726 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.724+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:12.737 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.736+0000 7f06ca4fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:12.737 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.736+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:12.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.741+0000 7f06ca4fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:12.744 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.741+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:12.759 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.758+0000 7f06ca4fdec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:12.963 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.961+0000 7f06ca4fdec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T10:21:12.964 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.961+0000 7f06ca4fdec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T10:21:12.964 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.961+0000 7f06ca4fdec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:21:12.964 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.961+0000 7f06ca4fdec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:21:12.964 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.961+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:12.974 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.972+0000 7f06ca4fdec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:12.974 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.972+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:12.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.994+0000 7f06ca4fdec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:21:12.996 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:12.994+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:13.005 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.003+0000 7f06ca4fdec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:13.005 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.003+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:13.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.027+0000 7f06ca4fdec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:21:13.029 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.027+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:13.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.031+0000 7f06ca4fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:13.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.031+0000 7f06ca4fdec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:21:13.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.031+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:13.038 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.037+0000 7f06ca4fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:13.038 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.037+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:13.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.045+0000 7f06ca4fdec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:13.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.045+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:13.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.055+0000 7f06ca4fdec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:21:13.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.055+0000 7f06ca4fdec0 20 zonegroup default 2019-12-09T10:21:13.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.055+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:13.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.067+0000 7f06ca4fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:13.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.067+0000 7f06ca4fdec0 10 Cannot find current period zone using local zone 2019-12-09T10:21:13.069 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.067+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:13.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.074+0000 7f06ca4fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:13.075 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.074+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:13.086 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.085+0000 7f06ca4fdec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:13.087 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.085+0000 7f06ca4fdec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:13.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.095+0000 7f06ca4fdec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:21:13.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.095+0000 7f06ca4fdec0 20 zone default found 2019-12-09T10:21:13.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.095+0000 7f06ca4fdec0 20 started sync module instance, tier type = 2019-12-09T10:21:13.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.095+0000 7f06ca4fdec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T10:21:13.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.463+0000 7f06ca4fdec0 20 add_watcher() i=0 2019-12-09T10:21:13.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.463+0000 7f06ca4fdec0 20 add_watcher() i=1 2019-12-09T10:21:13.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.463+0000 7f06ca4fdec0 20 add_watcher() i=2 2019-12-09T10:21:13.465 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.463+0000 7f06ca4fdec0 20 add_watcher() i=3 2019-12-09T10:21:13.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.463+0000 7f06ca4fdec0 20 add_watcher() i=4 2019-12-09T10:21:13.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.463+0000 7f06ca4fdec0 20 add_watcher() i=5 2019-12-09T10:21:13.466 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.463+0000 7f06ca4fdec0 20 add_watcher() i=6 2019-12-09T10:21:13.502 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.500+0000 7f06ca4fdec0 20 add_watcher() i=7 2019-12-09T10:21:13.502 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.500+0000 7f06ca4fdec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:21:13.502 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.500+0000 7f0687fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:21:13.513 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.511+0000 7f06b6f26700 0 -- 172.21.15.35:0/2108250898 >> v2:172.21.15.35:3300/0 conn(0x5633e12a77d0 msgr2=0x5633e12a9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:21:13.553 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:13.551+0000 7f06b7727700 0 -- 172.21.15.35:0/3687612324 >> [v2:172.21.15.22:6800/8484,v1:172.21.15.22:6801/8484] conn(0x7f05f8022490 msgr2=0x7f05f80249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:21:15.331 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.329+0000 7f05f67fc700 20 reqs_thread_entry: start 2019-12-09T10:21:15.331 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.329+0000 7f05effff700 20 reqs_thread_entry: start 2019-12-09T10:21:15.331 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.329+0000 7f06ca4fdec0 20 init_complete bucket index max shards: 0 2019-12-09T10:21:15.332 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.329+0000 7f05ee7fc700 20 reqs_thread_entry: start 2019-12-09T10:21:15.332 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.329+0000 7f06ca4fdec0 20 get_system_obj_state: rctx=0x7fff66ad3ba8 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x5633e1325520 s->prefetch_data=0 2019-12-09T10:21:15.332 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.329+0000 7f06ca4fdec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T10:21:15.337 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.335+0000 7f06ca4fdec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2019-12-09T10:21:15.337 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.335+0000 7f06ca4fdec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:21:15.337 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.335+0000 7f06ca4fdec0 20 get_system_obj_state: rctx=0x7fff66ad3bb8 obj=default.rgw.meta:users.email:testx$tenanteduser.client.0+test@test.test state=0x5633e13272d0 s->prefetch_data=0 2019-12-09T10:21:15.338 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.335+0000 7f06ca4fdec0 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test : miss 2019-12-09T10:21:15.343 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.341+0000 7f06ca4fdec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x0 2019-12-09T10:21:15.343 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.341+0000 7f06ca4fdec0 10 adding default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T10:21:15.343 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.341+0000 7f06ca4fdec0 20 get_system_obj_state: rctx=0x7fff66ad3bb8 obj=default.rgw.meta:users.keys:YTABYFXHGYOWVVUVVLWN state=0x5633e13272d0 s->prefetch_data=0 2019-12-09T10:21:15.344 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.341+0000 7f06ca4fdec0 10 cache get: name=default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN : miss 2019-12-09T10:21:15.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.347+0000 7f06ca4fdec0 10 cache put: name=default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN info.flags=0x0 2019-12-09T10:21:15.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.347+0000 7f06ca4fdec0 10 adding default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN to cache LRU end 2019-12-09T10:21:15.349 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.347+0000 7f06ca4fdec0 20 get_system_obj_state: rctx=0x7fff66ad37c8 obj=default.rgw.meta:users.keys:YTABYFXHGYOWVVUVVLWN state=0x5633e13272d0 s->prefetch_data=0 2019-12-09T10:21:15.350 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.347+0000 7f06ca4fdec0 10 cache get: name=default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN : type miss (requested=0x6, cached=0x0) 2019-12-09T10:21:15.355 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.353+0000 7f06ca4fdec0 10 cache put: name=default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN info.flags=0x0 2019-12-09T10:21:15.355 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.353+0000 7f06ca4fdec0 10 moving default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN to cache LRU end 2019-12-09T10:21:15.355 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.353+0000 7f06ca4fdec0 20 get_system_obj_state: rctx=0x7fff66ad3948 obj=default.rgw.meta:users.keys:YTABYFXHGYOWVVUVVLWN state=0x5633e13272d0 s->prefetch_data=0 2019-12-09T10:21:15.355 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.353+0000 7f06ca4fdec0 10 cache get: name=default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN : type miss (requested=0x6, cached=0x0) 2019-12-09T10:21:15.360 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.359+0000 7f06ca4fdec0 10 cache put: name=default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN info.flags=0x0 2019-12-09T10:21:15.361 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.359+0000 7f06ca4fdec0 10 moving default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN to cache LRU end 2019-12-09T10:21:15.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.420+0000 7f06ca4fdec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T10:21:15.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.420+0000 7f06ca4fdec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:21:15.422 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.420+0000 7f06ca4fdec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=691 2019-12-09T10:21:15.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.430+0000 7f06acff9700 10 RGWWatcher::handle_notify() notify_id 588410519630 cookie 94780820875776 notifier 4972 bl.length()=691 2019-12-09T10:21:15.432 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.430+0000 7f06acff9700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T10:21:15.432 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.430+0000 7f06acff9700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:21:15.572 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.568+0000 7f06ca4fdec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-12-09T10:21:15.572 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.568+0000 7f06ca4fdec0 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T10:21:15.572 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.568+0000 7f06ca4fdec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=224 2019-12-09T10:21:15.588 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.587+0000 7f06acff9700 10 RGWWatcher::handle_notify() notify_id 588410519603 cookie 94780820877104 notifier 4972 bl.length()=224 2019-12-09T10:21:15.589 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.587+0000 7f06acff9700 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-12-09T10:21:15.589 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:15.587+0000 7f06acff9700 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T10:21:16.517 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.513+0000 7f06ca4fdec0 10 cache put: name=default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN info.flags=0x7 2019-12-09T10:21:16.518 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.513+0000 7f06ca4fdec0 10 moving default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN to cache LRU end 2019-12-09T10:21:16.518 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.513+0000 7f06ca4fdec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=201 2019-12-09T10:21:16.545 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.543+0000 7f06acff9700 10 RGWWatcher::handle_notify() notify_id 588410519584 cookie 94780820857952 notifier 4972 bl.length()=201 2019-12-09T10:21:16.546 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.543+0000 7f06acff9700 10 cache put: name=default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN info.flags=0x7 2019-12-09T10:21:16.546 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.543+0000 7f06acff9700 10 moving default.rgw.meta+users.keys+YTABYFXHGYOWVVUVVLWN to cache LRU end 2019-12-09T10:21:16.641 INFO:teuthology.orchestra.run.smithi035.stdout:{ 2019-12-09T10:21:16.641 INFO:teuthology.orchestra.run.smithi035.stdout: "user_id": "testx$tenanteduser.client.0", 2019-12-09T10:21:16.641 INFO:teuthology.orchestra.run.smithi035.stdout: "display_name": "Mr. testx$tenanteduser.client.0", 2019-12-09T10:21:16.641 INFO:teuthology.orchestra.run.smithi035.stdout: "email": "testx$tenanteduser.client.0+test@test.test", 2019-12-09T10:21:16.641 INFO:teuthology.orchestra.run.smithi035.stdout: "suspended": 0, 2019-12-09T10:21:16.641 INFO:teuthology.orchestra.run.smithi035.stdout: "max_buckets": 1000, 2019-12-09T10:21:16.642 INFO:teuthology.orchestra.run.smithi035.stdout: "subusers": [], 2019-12-09T10:21:16.642 INFO:teuthology.orchestra.run.smithi035.stdout: "keys": [ 2019-12-09T10:21:16.642 INFO:teuthology.orchestra.run.smithi035.stdout: { 2019-12-09T10:21:16.642 INFO:teuthology.orchestra.run.smithi035.stdout: "user": "testx$tenanteduser.client.0", 2019-12-09T10:21:16.642 INFO:teuthology.orchestra.run.smithi035.stdout: "access_key": "YTABYFXHGYOWVVUVVLWN", 2019-12-09T10:21:16.643 INFO:teuthology.orchestra.run.smithi035.stdout: "secret_key": "PkCoQQpWjZPyn8l2m8IQAdZ+NJfZn7/uJWMkGTMxMGBi9JRHsJVscw==" 2019-12-09T10:21:16.643 INFO:teuthology.orchestra.run.smithi035.stdout: } 2019-12-09T10:21:16.643 INFO:teuthology.orchestra.run.smithi035.stdout: ], 2019-12-09T10:21:16.643 INFO:teuthology.orchestra.run.smithi035.stdout: "swift_keys": [], 2019-12-09T10:21:16.643 INFO:teuthology.orchestra.run.smithi035.stdout: "caps": [], 2019-12-09T10:21:16.643 INFO:teuthology.orchestra.run.smithi035.stdout: "op_mask": "read, write, delete", 2019-12-09T10:21:16.643 INFO:teuthology.orchestra.run.smithi035.stdout: "default_placement": "", 2019-12-09T10:21:16.643 INFO:teuthology.orchestra.run.smithi035.stdout: "default_storage_class": "", 2019-12-09T10:21:16.644 INFO:teuthology.orchestra.run.smithi035.stdout: "placement_tags": [], 2019-12-09T10:21:16.644 INFO:teuthology.orchestra.run.smithi035.stdout: "bucket_quota": { 2019-12-09T10:21:16.644 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T10:21:16.644 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T10:21:16.644 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T10:21:16.644 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T10:21:16.644 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T10:21:16.645 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T10:21:16.645 INFO:teuthology.orchestra.run.smithi035.stdout: "user_quota": { 2019-12-09T10:21:16.645 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T10:21:16.645 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T10:21:16.645 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T10:21:16.645 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T10:21:16.645 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T10:21:16.645 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T10:21:16.646 INFO:teuthology.orchestra.run.smithi035.stdout: "temp_url_keys": [], 2019-12-09T10:21:16.646 INFO:teuthology.orchestra.run.smithi035.stdout: "type": "rgw", 2019-12-09T10:21:16.646 INFO:teuthology.orchestra.run.smithi035.stdout: "mfa_ids": [] 2019-12-09T10:21:16.646 INFO:teuthology.orchestra.run.smithi035.stdout:} 2019-12-09T10:21:16.646 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:16.706 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.704+0000 7f06ca4fdec0 20 remove_watcher() i=0 2019-12-09T10:21:16.706 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.704+0000 7f06ca4fdec0 2 removed watcher, disabling cache 2019-12-09T10:21:16.771 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.769+0000 7f06ca4fdec0 20 remove_watcher() i=1 2019-12-09T10:21:16.809 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.807+0000 7f06ca4fdec0 20 remove_watcher() i=2 2019-12-09T10:21:16.879 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.876+0000 7f06ca4fdec0 20 remove_watcher() i=3 2019-12-09T10:21:16.925 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.923+0000 7f06ca4fdec0 20 remove_watcher() i=4 2019-12-09T10:21:16.964 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:16.959+0000 7f06ca4fdec0 20 remove_watcher() i=5 2019-12-09T10:21:17.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.009+0000 7f06ca4fdec0 20 remove_watcher() i=6 2019-12-09T10:21:17.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.045+0000 7f06ca4fdec0 20 remove_watcher() i=7 2019-12-09T10:21:17.058 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 7284367470 --totp-seed D44SQICEWJBADHQTJD2JNYDGHJLZKZFWYZZZR5X27LDMMEYRJLIQG7U25LQ6ZTNV --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T10:21:17.219 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.217+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.251 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.248+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:17.251 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.248+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.263 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.261+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:17.263 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.261+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.270 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.268+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:17.271 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.268+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.281+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:17.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.507+0000 7f0b3d0f8ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T10:21:17.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.507+0000 7f0b3d0f8ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T10:21:17.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.507+0000 7f0b3d0f8ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:21:17.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.507+0000 7f0b3d0f8ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:21:17.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.507+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.522 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.520+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:17.523 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.520+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.545 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.544+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:21:17.546 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.544+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.562 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.560+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:17.562 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.560+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.579 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.577+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:21:17.579 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.577+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.587 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.585+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:17.587 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.585+0000 7f0b3d0f8ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:21:17.588 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.585+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.595 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.593+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:17.595 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.593+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.606+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:17.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.606+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.626 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.625+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:21:17.627 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.625+0000 7f0b3d0f8ec0 20 zonegroup default 2019-12-09T10:21:17.627 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.625+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.633 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.631+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:17.633 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.631+0000 7f0b3d0f8ec0 10 Cannot find current period zone using local zone 2019-12-09T10:21:17.633 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.631+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.641 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.639+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:21:17.641 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.639+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.654 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.652+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:21:17.654 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.652+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:17.671 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.668+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:21:17.671 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.669+0000 7f0b3d0f8ec0 20 zone default found 2019-12-09T10:21:17.671 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.669+0000 7f0b3d0f8ec0 20 started sync module instance, tier type = 2019-12-09T10:21:17.671 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:17.669+0000 7f0b3d0f8ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T10:21:18.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.043+0000 7f0b3d0f8ec0 20 add_watcher() i=0 2019-12-09T10:21:18.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.043+0000 7f0b3d0f8ec0 20 add_watcher() i=1 2019-12-09T10:21:18.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.043+0000 7f0b3d0f8ec0 20 add_watcher() i=2 2019-12-09T10:21:18.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.043+0000 7f0b3d0f8ec0 20 add_watcher() i=3 2019-12-09T10:21:18.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.043+0000 7f0b3d0f8ec0 20 add_watcher() i=4 2019-12-09T10:21:18.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.043+0000 7f0b3d0f8ec0 20 add_watcher() i=5 2019-12-09T10:21:18.073 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.071+0000 7f0b3d0f8ec0 20 add_watcher() i=6 2019-12-09T10:21:18.094 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.091+0000 7f0b3d0f8ec0 20 add_watcher() i=7 2019-12-09T10:21:18.094 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.091+0000 7f0b3d0f8ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:21:18.094 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.091+0000 7f0afa7e4700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:21:18.097 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.095+0000 7f0b2a322700 0 -- 172.21.15.35:0/595317340 >> v2:172.21.15.35:3300/0 conn(0x55a5952eb130 msgr2=0x55a5952f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:21:18.128 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.126+0000 7f0b2ab23700 0 -- 172.21.15.35:0/2646073903 >> v2:172.21.15.22:3300/0 conn(0x55a5952f00c0 msgr2=0x55a5952f04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T10:21:18.331 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:18.329+0000 7f0b2ab23700 0 --2- 172.21.15.35:0/2646073903 >> v2:172.21.15.22:3300/0 conn(0x55a5952f00c0 0x55a5952f04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T10:21:20.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.075+0000 7f0a70ff9700 20 reqs_thread_entry: start 2019-12-09T10:21:20.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.075+0000 7f0a6a7fc700 20 reqs_thread_entry: start 2019-12-09T10:21:20.077 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.075+0000 7f0b3d0f8ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:21:20.078 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.076+0000 7f0a68ff9700 20 reqs_thread_entry: start 2019-12-09T10:21:20.078 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.076+0000 7f0b3d0f8ec0 20 get_system_obj_state: rctx=0x7fff718a4de8 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x55a595364e10 s->prefetch_data=0 2019-12-09T10:21:20.078 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.076+0000 7f0b3d0f8ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T10:21:20.094 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.092+0000 7f0b3d0f8ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-12-09T10:21:20.095 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.092+0000 7f0b3d0f8ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:21:20.095 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.092+0000 7f0b3d0f8ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:21:20.095 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.092+0000 7f0b3d0f8ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:21:20.095 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.092+0000 7f0b3d0f8ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T10:21:20.095 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.092+0000 7f0b3d0f8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:21:20.116 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.114+0000 7f0b3d0f8ec0 20 rados_obj.operate() r=0 bl.length=491 2019-12-09T10:21:20.116 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.114+0000 7f0b3d0f8ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-12-09T10:21:20.116 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.114+0000 7f0b3d0f8ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:21:20.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.283+0000 7f0b3d0f8ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T10:21:20.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.283+0000 7f0b3d0f8ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:21:20.286 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.283+0000 7f0b3d0f8ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=705 2019-12-09T10:21:20.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.295+0000 7f0b0f7fe700 10 RGWWatcher::handle_notify() notify_id 588410519631 cookie 94169660761456 notifier 4990 bl.length()=705 2019-12-09T10:21:20.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.295+0000 7f0b0f7fe700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T10:21:20.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.295+0000 7f0b0f7fe700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:21:20.807 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.805+0000 7f0b3d0f8ec0 20 remove_watcher() i=0 2019-12-09T10:21:20.807 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.805+0000 7f0b3d0f8ec0 2 removed watcher, disabling cache 2019-12-09T10:21:20.841 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.839+0000 7f0b3d0f8ec0 20 remove_watcher() i=1 2019-12-09T10:21:20.871 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.870+0000 7f0b3d0f8ec0 20 remove_watcher() i=2 2019-12-09T10:21:20.934 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.932+0000 7f0b3d0f8ec0 20 remove_watcher() i=3 2019-12-09T10:21:20.974 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:20.972+0000 7f0b3d0f8ec0 20 remove_watcher() i=4 2019-12-09T10:21:21.026 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:21.024+0000 7f0b3d0f8ec0 20 remove_watcher() i=5 2019-12-09T10:21:21.096 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:21.093+0000 7f0b3d0f8ec0 20 remove_watcher() i=6 2019-12-09T10:21:21.158 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T10:21:21.156+0000 7f0b3d0f8ec0 20 remove_watcher() i=7 2019-12-09T10:21:21.168 INFO:tasks.s3tests:Configuring s3-tests... 2019-12-09T10:21:21.168 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest/s3-tests && ./bootstrap 2019-12-09T10:21:26.813 INFO:teuthology.orchestra.run.smithi035.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-12-09T10:21:26.922 INFO:teuthology.orchestra.run.smithi035.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T10:21:26.962 INFO:teuthology.orchestra.run.smithi035.stdout:Loading mirror speeds from cached hostfile 2019-12-09T10:21:26.963 INFO:teuthology.orchestra.run.smithi035.stdout: * base: repos-va.psychz.net 2019-12-09T10:21:26.963 INFO:teuthology.orchestra.run.smithi035.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T10:21:26.964 INFO:teuthology.orchestra.run.smithi035.stdout: * extras: mirror.math.princeton.edu 2019-12-09T10:21:26.964 INFO:teuthology.orchestra.run.smithi035.stdout: * updates: mirror.cogentco.com 2019-12-09T10:21:27.643 INFO:teuthology.orchestra.run.smithi035.stdout:7 packages excluded due to repository priority protections 2019-12-09T10:21:27.997 INFO:teuthology.orchestra.run.smithi035.stdout:Resolving Dependencies 2019-12-09T10:21:27.997 INFO:teuthology.orchestra.run.smithi035.stdout:--> Running transaction check 2019-12-09T10:21:27.998 INFO:teuthology.orchestra.run.smithi035.stdout:---> Package libffi-devel.x86_64 0:3.0.13-18.el7 will be installed 2019-12-09T10:21:28.404 INFO:teuthology.orchestra.run.smithi035.stdout:--> Finished Dependency Resolution 2019-12-09T10:21:28.647 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:28.647 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies Resolved 2019-12-09T10:21:28.647 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:28.648 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T10:21:28.648 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2019-12-09T10:21:28.648 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T10:21:28.648 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2019-12-09T10:21:28.648 INFO:teuthology.orchestra.run.smithi035.stdout: libffi-devel x86_64 3.0.13-18.el7 base 23 k 2019-12-09T10:21:28.648 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:28.649 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2019-12-09T10:21:28.649 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2019-12-09T10:21:28.649 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2019-12-09T10:21:28.649 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:28.649 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 23 k 2019-12-09T10:21:28.649 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 27 k 2019-12-09T10:21:28.649 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading packages: 2019-12-09T10:21:28.750 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2019-12-09T10:21:28.753 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2019-12-09T10:21:28.754 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded 2019-12-09T10:21:28.755 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2019-12-09T10:21:29.406 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-12-09T10:21:29.776 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-12-09T10:21:29.777 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:29.777 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2019-12-09T10:21:29.777 INFO:teuthology.orchestra.run.smithi035.stdout: libffi-devel.x86_64 0:3.0.13-18.el7 2019-12-09T10:21:29.777 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:29.777 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2019-12-09T10:21:29.980 INFO:teuthology.orchestra.run.smithi035.stdout:New python executable in /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python 2019-12-09T10:21:31.579 INFO:teuthology.orchestra.run.smithi035.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T10:21:31.872 INFO:teuthology.orchestra.run.smithi035.stderr:Cache entry deserialization failed, entry ignored 2019-12-09T10:21:32.019 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting pip 2019-12-09T10:21:32.100 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-12-09T10:21:32.168 INFO:teuthology.orchestra.run.smithi035.stdout:Installing collected packages: pip 2019-12-09T10:21:32.169 INFO:teuthology.orchestra.run.smithi035.stdout: Found existing installation: pip 9.0.1 2019-12-09T10:21:32.229 INFO:teuthology.orchestra.run.smithi035.stdout: Uninstalling pip-9.0.1: 2019-12-09T10:21:32.275 INFO:teuthology.orchestra.run.smithi035.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T10:21:32.912 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully installed pip-19.3.1 2019-12-09T10:21:33.186 INFO:teuthology.orchestra.run.smithi035.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T10:21:33.772 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting setuptools==32.3.1 2019-12-09T10:21:33.775 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-12-09T10:21:33.819 INFO:teuthology.orchestra.run.smithi035.stdout:Installing collected packages: setuptools 2019-12-09T10:21:33.819 INFO:teuthology.orchestra.run.smithi035.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T10:21:33.835 INFO:teuthology.orchestra.run.smithi035.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T10:21:33.844 INFO:teuthology.orchestra.run.smithi035.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T10:21:33.978 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully installed setuptools-32.3.1 2019-12-09T10:21:34.433 INFO:teuthology.orchestra.run.smithi035.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T10:21:34.674 INFO:teuthology.orchestra.run.smithi035.stdout:Processing /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67/PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl 2019-12-09T10:21:34.742 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting nose>=1.0.0 2019-12-09T10:21:34.744 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-12-09T10:21:34.841 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting boto>=2.6.0 2019-12-09T10:21:34.848 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-12-09T10:21:35.554 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting boto3>=1.0.0 2019-12-09T10:21:35.600 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/1f/a5/6b25e39aea40bc03163615c07572d90aaddc9951f5af9c21204d3da46398/boto3-1.10.34-py2.py3-none-any.whl (128kB) 2019-12-09T10:21:35.727 INFO:teuthology.orchestra.run.smithi035.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-12-09T10:21:35.985 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting gevent>=1.0 2019-12-09T10:21:35.999 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T10:21:36.259 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting isodate>=0.4.4 2019-12-09T10:21:36.270 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl (45kB) 2019-12-09T10:21:36.444 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting requests>=0.14.0 2019-12-09T10:21:36.455 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB) 2019-12-09T10:21:36.775 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting pytz>=2011k 2019-12-09T10:21:36.786 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB) 2019-12-09T10:21:37.006 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting ordereddict 2019-12-09T10:21:37.016 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/53/25/ef88e8e45db141faa9598fbf7ad0062df8f50f881a36ed6a0073e1572126/ordereddict-1.1.tar.gz 2019-12-09T10:21:37.396 INFO:teuthology.orchestra.run.smithi035.stdout:Processing /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488/httplib2-0.14.0-cp27-none-any.whl 2019-12-09T10:21:37.819 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting lxml 2019-12-09T10:21:37.835 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T10:21:38.042 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting jmespath<1.0.0,>=0.7.1 2019-12-09T10:21:38.053 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl 2019-12-09T10:21:38.690 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting botocore<1.14.0,>=1.13.34 2019-12-09T10:21:38.700 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/59/cb/b4772a4abc128f6fd637af9d7fc93a1db11617859af680fc2b2e6282eb95/botocore-1.13.34-py2.py3-none-any.whl (5.8MB) 2019-12-09T10:21:39.341 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting s3transfer<0.3.0,>=0.2.0 2019-12-09T10:21:39.351 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl (70kB) 2019-12-09T10:21:39.593 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T10:21:39.595 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T10:21:39.663 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting six 2019-12-09T10:21:39.673 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-12-09T10:21:39.773 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting chardet<3.1.0,>=3.0.2 2019-12-09T10:21:39.785 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) 2019-12-09T10:21:39.886 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting idna<2.9,>=2.5 2019-12-09T10:21:39.896 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB) 2019-12-09T10:21:40.082 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 2019-12-09T10:21:40.094 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB) 2019-12-09T10:21:40.254 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting certifi>=2017.4.17 2019-12-09T10:21:40.262 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB) 2019-12-09T10:21:40.378 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting docutils<0.16,>=0.10 2019-12-09T10:21:40.390 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/3a/dc/bf2b15d1fa15a6f7a9e77a61b74ecbbae7258558fcda8ffc9a6638a6b327/docutils-0.15.2-py2-none-any.whl (548kB) 2019-12-09T10:21:40.604 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting python-dateutil<2.8.1,>=2.1; python_version >= "2.7" 2019-12-09T10:21:40.615 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB) 2019-12-09T10:21:40.743 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" 2019-12-09T10:21:40.754 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl 2019-12-09T10:21:40.791 INFO:teuthology.orchestra.run.smithi035.stdout:Building wheels for collected packages: ordereddict 2019-12-09T10:21:40.792 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for ordereddict (setup.py): started 2019-12-09T10:21:41.017 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for ordereddict (setup.py): finished with status 'done' 2019-12-09T10:21:41.018 INFO:teuthology.orchestra.run.smithi035.stdout: Created wheel for ordereddict: filename=ordereddict-1.1-cp27-none-any.whl size=3327 sha256=d3945f8888e1229b31cb507204bb9c3adbe23f555911cff4cba2bda40c8c3385 2019-12-09T10:21:41.018 INFO:teuthology.orchestra.run.smithi035.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/78/d3/a8/9f52c7a389a0ffba6f575a3886b5f7a55461fb6ec34aa5fd38 2019-12-09T10:21:41.022 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully built ordereddict 2019-12-09T10:21:41.130 INFO:teuthology.orchestra.run.smithi035.stdout:Installing collected packages: PyYAML, nose, boto, jmespath, docutils, six, python-dateutil, urllib3, botocore, futures, s3transfer, boto3, bunch, greenlet, gevent, isodate, chardet, idna, certifi, requests, pytz, ordereddict, httplib2, lxml 2019-12-09T10:21:43.524 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully installed PyYAML-5.2 boto-2.49.0 boto3-1.10.34 botocore-1.13.34 bunch-1.0.1 certifi-2019.11.28 chardet-3.0.4 docutils-0.15.2 futures-3.3.0 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 idna-2.8 isodate-0.6.0 jmespath-0.9.4 lxml-4.4.2 nose-1.3.7 ordereddict-1.1 python-dateutil-2.8.0 pytz-2019.3 requests-2.22.0 s3transfer-0.2.1 six-1.13.0 urllib3-1.25.7 2019-12-09T10:21:43.916 INFO:teuthology.orchestra.run.smithi035.stdout:running develop 2019-12-09T10:21:43.969 INFO:teuthology.orchestra.run.smithi035.stdout:running egg_info 2019-12-09T10:21:43.969 INFO:teuthology.orchestra.run.smithi035.stdout:creating s3tests.egg-info 2019-12-09T10:21:43.969 INFO:teuthology.orchestra.run.smithi035.stdout:writing requirements to s3tests.egg-info/requires.txt 2019-12-09T10:21:43.969 INFO:teuthology.orchestra.run.smithi035.stdout:writing s3tests.egg-info/PKG-INFO 2019-12-09T10:21:43.970 INFO:teuthology.orchestra.run.smithi035.stdout:writing top-level names to s3tests.egg-info/top_level.txt 2019-12-09T10:21:43.970 INFO:teuthology.orchestra.run.smithi035.stdout:writing dependency_links to s3tests.egg-info/dependency_links.txt 2019-12-09T10:21:43.971 INFO:teuthology.orchestra.run.smithi035.stdout:writing entry points to s3tests.egg-info/entry_points.txt 2019-12-09T10:21:43.971 INFO:teuthology.orchestra.run.smithi035.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T10:21:43.972 INFO:teuthology.orchestra.run.smithi035.stdout:reading manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T10:21:43.973 INFO:teuthology.orchestra.run.smithi035.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T10:21:43.975 INFO:teuthology.orchestra.run.smithi035.stdout:running build_ext 2019-12-09T10:21:43.975 INFO:teuthology.orchestra.run.smithi035.stdout:Creating /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/s3tests.egg-link (link to .) 2019-12-09T10:21:43.975 INFO:teuthology.orchestra.run.smithi035.stdout:Adding s3tests 0.0.1 to easy-install.pth file 2019-12-09T10:21:43.978 INFO:teuthology.orchestra.run.smithi035.stdout:Installing s3tests-fuzz-headers script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T10:21:43.978 INFO:teuthology.orchestra.run.smithi035.stdout:Installing s3tests-generate-objects script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T10:21:43.978 INFO:teuthology.orchestra.run.smithi035.stdout:Installing s3tests-test-roundtrip script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T10:21:43.978 INFO:teuthology.orchestra.run.smithi035.stdout:Installing s3tests-analysis-rwstats script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T10:21:43.978 INFO:teuthology.orchestra.run.smithi035.stdout:Installing s3tests-test-readwrite script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T10:21:43.978 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:43.979 INFO:teuthology.orchestra.run.smithi035.stdout:Installed /home/ubuntu/cephtest/s3-tests 2019-12-09T10:21:43.979 INFO:teuthology.orchestra.run.smithi035.stdout:Processing dependencies for s3tests==0.0.1 2019-12-09T10:21:44.028 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for isodate==0.6.0 2019-12-09T10:21:44.029 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: isodate 0.6.0 2019-12-09T10:21:44.029 INFO:teuthology.orchestra.run.smithi035.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-12-09T10:21:44.029 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.029 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.030 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for gevent==1.4.0 2019-12-09T10:21:44.030 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: gevent 1.4.0 2019-12-09T10:21:44.030 INFO:teuthology.orchestra.run.smithi035.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-12-09T10:21:44.030 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.030 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.031 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for bunch==1.0.1 2019-12-09T10:21:44.032 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: bunch 1.0.1 2019-12-09T10:21:44.032 INFO:teuthology.orchestra.run.smithi035.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-12-09T10:21:44.032 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.032 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.032 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for PyYAML==5.2 2019-12-09T10:21:44.032 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: PyYAML 5.2 2019-12-09T10:21:44.033 INFO:teuthology.orchestra.run.smithi035.stdout:Adding PyYAML 5.2 to easy-install.pth file 2019-12-09T10:21:44.033 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.033 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.034 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for boto3==1.10.34 2019-12-09T10:21:44.034 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: boto3 1.10.34 2019-12-09T10:21:44.034 INFO:teuthology.orchestra.run.smithi035.stdout:Adding boto3 1.10.34 to easy-install.pth file 2019-12-09T10:21:44.034 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.034 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.034 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for boto==2.49.0 2019-12-09T10:21:44.036 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: boto 2.49.0 2019-12-09T10:21:44.036 INFO:teuthology.orchestra.run.smithi035.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-12-09T10:21:44.036 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.036 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.037 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for six==1.13.0 2019-12-09T10:21:44.037 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: six 1.13.0 2019-12-09T10:21:44.037 INFO:teuthology.orchestra.run.smithi035.stdout:Adding six 1.13.0 to easy-install.pth file 2019-12-09T10:21:44.037 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.037 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.037 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for greenlet==0.4.15 2019-12-09T10:21:44.037 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: greenlet 0.4.15 2019-12-09T10:21:44.038 INFO:teuthology.orchestra.run.smithi035.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-12-09T10:21:44.038 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.038 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.038 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for s3transfer==0.2.1 2019-12-09T10:21:44.038 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: s3transfer 0.2.1 2019-12-09T10:21:44.038 INFO:teuthology.orchestra.run.smithi035.stdout:Adding s3transfer 0.2.1 to easy-install.pth file 2019-12-09T10:21:44.038 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.038 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.039 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for botocore==1.13.34 2019-12-09T10:21:44.039 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: botocore 1.13.34 2019-12-09T10:21:44.040 INFO:teuthology.orchestra.run.smithi035.stdout:Adding botocore 1.13.34 to easy-install.pth file 2019-12-09T10:21:44.040 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.040 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.040 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for jmespath==0.9.4 2019-12-09T10:21:44.041 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: jmespath 0.9.4 2019-12-09T10:21:44.042 INFO:teuthology.orchestra.run.smithi035.stdout:Adding jmespath 0.9.4 to easy-install.pth file 2019-12-09T10:21:44.042 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.042 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.042 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for futures==3.3.0 2019-12-09T10:21:44.042 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: futures 3.3.0 2019-12-09T10:21:44.042 INFO:teuthology.orchestra.run.smithi035.stdout:Adding futures 3.3.0 to easy-install.pth file 2019-12-09T10:21:44.043 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.043 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.043 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for urllib3==1.25.7 2019-12-09T10:21:44.043 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: urllib3 1.25.7 2019-12-09T10:21:44.043 INFO:teuthology.orchestra.run.smithi035.stdout:Adding urllib3 1.25.7 to easy-install.pth file 2019-12-09T10:21:44.043 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.043 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.043 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for python-dateutil==2.8.0 2019-12-09T10:21:44.044 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: python-dateutil 2.8.0 2019-12-09T10:21:44.044 INFO:teuthology.orchestra.run.smithi035.stdout:Adding python-dateutil 2.8.0 to easy-install.pth file 2019-12-09T10:21:44.045 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.045 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.045 INFO:teuthology.orchestra.run.smithi035.stdout:Searching for docutils==0.15.2 2019-12-09T10:21:44.045 INFO:teuthology.orchestra.run.smithi035.stdout:Best match: docutils 0.15.2 2019-12-09T10:21:44.045 INFO:teuthology.orchestra.run.smithi035.stdout:Adding docutils 0.15.2 to easy-install.pth file 2019-12-09T10:21:44.045 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T10:21:44.046 INFO:teuthology.orchestra.run.smithi035.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:21:44.046 INFO:teuthology.orchestra.run.smithi035.stdout:Finished processing dependencies for s3tests==0.0.1 2019-12-09T10:21:44.053 INFO:teuthology.orchestra.run.smithi035:> cat > /home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2019-12-09T10:21:44.107 INFO:tasks.s3tests:Configuring boto... 2019-12-09T10:21:44.107 INFO:teuthology.orchestra.run.smithi035:> cat > /home/ubuntu/cephtest/boto.cfg 2019-12-09T10:21:44.198 INFO:teuthology.orchestra.run.smithi035:s3 tests against rgw> S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain' 2019-12-09T10:21:51.393 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_none ... ok 2019-12-09T10:21:55.299 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_empty ... ok 2019-12-09T10:21:58.934 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_date_and_amz_date ... ok 2019-12-09T10:22:02.060 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_amz_date_and_no_date ... ok 2019-12-09T10:22:05.966 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_none ... ok 2019-12-09T10:22:08.177 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_contentlength_none ... ok 2019-12-09T10:22:14.329 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_acl_create_contentlength_none ... ok 2019-12-09T10:22:15.995 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_contentlength_none ... ok 2019-12-09T10:22:16.173 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_empty ... ok 2019-12-09T10:22:16.307 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_none ... ok 2019-12-09T10:22:18.286 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws2 ... ok 2019-12-09T10:22:21.332 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws2 ... ok 2019-12-09T10:22:24.805 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws2 ... ok 2019-12-09T10:22:27.708 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws2 ... ok 2019-12-09T10:22:28.296 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws2 ... ok 2019-12-09T10:22:28.584 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws2 ... ok 2019-12-09T10:22:28.584 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws4 ... SKIP 2019-12-09T10:22:28.584 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws4 ... SKIP 2019-12-09T10:22:28.585 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws4 ... SKIP 2019-12-09T10:22:28.585 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws4 ... SKIP 2019-12-09T10:22:28.585 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_empty_aws4 ... SKIP 2019-12-09T10:22:28.585 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_none_aws4 ... SKIP 2019-12-09T10:22:28.585 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_date_invalid_aws4 ... SKIP 2019-12-09T10:22:28.585 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_invalid_aws4 ... SKIP 2019-12-09T10:22:28.585 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_date_empty_aws4 ... SKIP 2019-12-09T10:22:28.586 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_empty_aws4 ... SKIP 2019-12-09T10:22:28.586 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws4 ... SKIP 2019-12-09T10:22:28.586 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_none_aws4 ... SKIP 2019-12-09T10:22:28.586 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_today_aws4 ... SKIP 2019-12-09T10:22:28.587 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_today_aws4 ... SKIP 2019-12-09T10:22:28.587 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_today_aws4 ... SKIP 2019-12-09T10:22:28.587 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_today_aws4 ... SKIP 2019-12-09T10:22:28.587 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_epoch_aws4 ... SKIP 2019-12-09T10:22:28.592 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-12-09T10:22:28.592 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_end_aws4 ... SKIP 2019-12-09T10:22:28.593 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_end_aws4 ... SKIP 2019-12-09T10:22:28.593 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_custom_header_aws4 ... SKIP 2019-12-09T10:22:28.593 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_header_aws4 ... SKIP 2019-12-09T10:22:28.593 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws4 ... SKIP 2019-12-09T10:22:28.593 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_empty_aws4 ... SKIP 2019-12-09T10:22:28.593 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_none_aws4 ... SKIP 2019-12-09T10:22:28.594 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_invalid_aws4 ... SKIP 2019-12-09T10:22:28.594 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_invalid_aws4 ... SKIP 2019-12-09T10:22:28.594 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_empty_aws4 ... SKIP 2019-12-09T10:22:28.594 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_empty_aws4 ... SKIP 2019-12-09T10:22:28.595 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws4 ... SKIP 2019-12-09T10:22:28.595 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_none_aws4 ... SKIP 2019-12-09T10:22:28.595 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_today_aws4 ... SKIP 2019-12-09T10:22:28.595 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_today_aws4 ... SKIP 2019-12-09T10:22:28.595 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_after_today_aws4 ... SKIP 2019-12-09T10:22:28.595 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_after_today_aws4 ... SKIP 2019-12-09T10:22:28.596 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws4 ... SKIP 2019-12-09T10:22:28.596 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-12-09T10:22:35.234 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_versioning_obj_read_not_exist_null ... ok 2019-12-09T10:22:35.234 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_lifecycle_transition ... SKIP 2019-12-09T10:22:35.234 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_lifecycle_transition_single_rule_multi_trans ... SKIP 2019-12-09T10:22:35.236 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_lifecycle_set_noncurrent_transition ... SKIP 2019-12-09T10:22:35.236 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_object_storage_class ... SKIP 2019-12-09T10:22:35.236 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_object_storage_class_multipart ... SKIP 2019-12-09T10:22:35.236 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_object_modify_storage_class ... SKIP 2019-12-09T10:22:35.236 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_object_modify_storage_class_multipart ... SKIP 2019-12-09T10:22:35.237 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_object_storage_class_copy ... SKIP 2019-12-09T10:22:35.237 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_object_storage_class_copy_multipart ... SKIP 2019-12-09T10:22:44.238 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_atomic_dual_conditional_write_1mb ... ok 2019-12-09T10:22:51.518 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_atomic_write_bucket_gone ... ok 2019-12-09T10:22:59.822 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_1 ... ok 2019-12-09T10:23:04.124 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_2 ... ok 2019-12-09T10:23:10.747 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_bucket_policy_set_condition_operator_end_with_IfExists ... ok 2019-12-09T10:23:15.548 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_enc ... ok 2019-12-09T10:23:19.038 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_request_obj_tag ... ok 2019-12-09T10:23:26.107 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.check_can_test_website ... SKIP 2019-12-09T10:23:26.107 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_nonexistant_bucket_rgw ... SKIP 2019-12-09T10:23:26.108 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_public_index ... SKIP 2019-12-09T10:23:26.108 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_public_index ... SKIP 2019-12-09T10:23:26.108 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty ... SKIP 2019-12-09T10:23:26.108 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty ... SKIP 2019-12-09T10:23:26.108 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index ... SKIP 2019-12-09T10:23:26.108 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index ... SKIP 2019-12-09T10:23:26.109 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_missingerrordoc ... SKIP 2019-12-09T10:23:26.109 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_missingerrordoc ... SKIP 2019-12-09T10:23:26.109 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_missingerrordoc ... SKIP 2019-12-09T10:23:26.109 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_missingerrordoc ... SKIP 2019-12-09T10:23:26.109 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_blockederrordoc ... SKIP 2019-12-09T10:23:26.109 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_blockederrordoc ... SKIP 2019-12-09T10:23:26.109 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_blockederrordoc ... SKIP 2019-12-09T10:23:26.111 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_blockederrordoc ... SKIP 2019-12-09T10:23:26.111 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_gooderrordoc ... SKIP 2019-12-09T10:23:26.111 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_gooderrordoc ... SKIP 2019-12-09T10:23:26.111 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_gooderrordoc ... SKIP 2019-12-09T10:23:26.111 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_gooderrordoc ... SKIP 2019-12-09T10:23:26.111 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_base ... SKIP 2019-12-09T10:23:26.113 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path ... SKIP 2019-12-09T10:23:26.113 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path_upgrade ... SKIP 2019-12-09T10:23:26.113 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_xredirect_nonwebsite ... SKIP 2019-12-09T10:23:26.114 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_relative ... SKIP 2019-12-09T10:23:26.114 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_abs ... SKIP 2019-12-09T10:23:26.114 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_relative ... SKIP 2019-12-09T10:23:26.114 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_abs ... SKIP 2019-12-09T10:23:26.114 INFO:teuthology.orchestra.run.smithi035.stderr:SKIP 2019-12-09T10:23:26.132 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.functional.test_utils.test_generate ... ok 2019-12-09T10:25:12.055 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-12-09T10:25:12.056 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-12-09T10:25:12.056 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-12-09T10:25:12.056 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-12-09T10:25:12.056 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-12-09T10:25:12.056 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-12-09T10:25:12.057 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-12-09T10:25:12.434 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-12-09T10:25:12.592 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-12-09T10:25:12.927 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-12-09T10:25:12.951 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-12-09T10:25:12.974 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-12-09T10:25:12.974 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-12-09T10:25:12.974 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-12-09T10:25:12.975 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-12-09T10:25:12.975 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-12-09T10:25:12.975 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-12-09T10:25:12.975 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-12-09T10:25:12.975 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-12-09T10:25:12.975 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-12-09T10:25:12.976 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-12-09T10:25:12.976 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-12-09T10:25:12.979 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-12-09T10:25:12.980 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-12-09T10:25:12.980 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-12-09T10:25:12.981 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_null_child ... ok 2019-12-09T10:25:12.985 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-12-09T10:25:12.985 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-12-09T10:25:12.985 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-12-09T10:25:12.985 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-12-09T10:25:13.011 INFO:teuthology.orchestra.run.smithi035.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-12-09T10:25:13.036 INFO:teuthology.orchestra.run.smithi035.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-12-09T10:25:13.037 INFO:teuthology.orchestra.run.smithi035.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-12-09T10:25:13.191 INFO:teuthology.orchestra.run.smithi035.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-12-09T10:25:17.749 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_short ... ok 2019-12-09T10:25:29.102 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_bad ... ok 2019-12-09T10:25:31.245 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_empty ... ok 2019-12-09T10:25:37.292 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_none ... ok 2019-12-09T10:25:42.282 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_mismatch ... ok 2019-12-09T10:25:47.082 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_empty ... ok 2019-12-09T10:25:52.495 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_none ... ok 2019-12-09T10:25:53.968 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:13:30.232 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:53.973 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:13:30.238 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:53.988 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:13:30.211 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:53.990 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:13:30.213 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:53.994 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:13:30.217 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:54.000 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:13:30.223 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:54.193 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:13:30.458 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:54.204 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:13:30.427 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:54.220 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:13:30.443 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:54.714 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_negative ... ok 2019-12-09T10:25:58.627 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_invalid ... ok 2019-12-09T10:26:01.314 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_empty ... ok 2019-12-09T10:26:03.253 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_none ... ok 2019-12-09T10:26:05.882 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_bucket_put_bad_canned_acl ... ok 2019-12-09T10:26:08.170 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_mismatch ... ok 2019-12-09T10:26:10.617 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_empty ... ok 2019-12-09T10:26:10.686 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_negative ... ok 2019-12-09T10:26:13.635 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws2 ... ok 2019-12-09T10:26:17.855 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_empty_aws2 ... ok 2019-12-09T10:26:23.703 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_none_aws2 ... ok 2019-12-09T10:26:25.784 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_invalid_aws2 ... ok 2019-12-09T10:26:30.323 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_empty_aws2 ... ok 2019-12-09T10:26:33.952 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_today_aws2 ... ok 2019-12-09T10:26:36.614 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_epoch_aws2 ... ok 2019-12-09T10:26:39.548 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_after_end_aws2 ... ok 2019-12-09T10:26:41.910 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_empty_aws2 ... ok 2019-12-09T10:26:45.350 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_none_aws2 ... ok 2019-12-09T10:26:45.443 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_invalid_aws2 ... ok 2019-12-09T10:26:45.690 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_empty_aws2 ... ok 2019-12-09T10:26:46.280 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_today_aws2 ... ok 2019-12-09T10:26:46.527 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_after_today_aws2 ... ok 2019-12-09T10:26:46.607 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws2 ... ok 2019-12-09T10:26:50.825 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_empty ... ok 2019-12-09T10:26:59.092 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_distinct ... ok 2019-12-09T10:27:06.853 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_many ... ok 2019-12-09T10:27:15.381 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_many ... ok 2019-12-09T10:27:21.862 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_basic_key_count ... ok 2019-12-09T10:27:30.648 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_basic ... ok 2019-12-09T10:27:37.830 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_basic ... ok 2019-12-09T10:27:52.963 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix ... ok 2019-12-09T10:28:06.814 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix ... ok 2019-12-09T10:28:10.268 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_ends_with_delimiter ... ok 2019-12-09T10:28:14.915 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_ends_with_delimiter ... ok 2019-12-09T10:28:23.421 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_alt ... ok 2019-12-09T10:28:30.548 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_alt ... ok 2019-12-09T10:28:32.638 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:16:08.903 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:28:32.640 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:16:08.863 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:28:32.645 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:16:08.869 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:28:32.648 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:16:08.912 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:28:32.676 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:16:08.900 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:28:32.684 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:16:08.907 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:28:32.735 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:16:08.958 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:28:32.782 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:16:09.005 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:28:32.810 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:16:09.074 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:28:42.405 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_underscore ... ok 2019-12-09T10:28:55.906 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_underscore ... ok 2019-12-09T10:29:03.517 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_percentage ... ok 2019-12-09T10:29:12.141 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_percentage ... ok 2019-12-09T10:29:20.688 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_whitespace ... ok 2019-12-09T10:29:26.835 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_whitespace ... ok 2019-12-09T10:29:34.286 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_dot ... ok 2019-12-09T10:29:42.740 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_dot ... ok 2019-12-09T10:29:49.502 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_unreadable ... ok 2019-12-09T10:29:57.469 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_unreadable ... ok 2019-12-09T10:30:02.533 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_empty ... ok 2019-12-09T10:30:08.732 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_empty ... ok 2019-12-09T10:30:17.151 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_none ... ok 2019-12-09T10:30:23.888 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_none ... ok 2019-12-09T10:30:29.832 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_notempty ... ok 2019-12-09T10:30:34.774 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_defaultempty ... ok 2019-12-09T10:30:41.325 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_empty ... ok 2019-12-09T10:30:47.810 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_exist ... ok 2019-12-09T10:30:56.669 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_not_exist ... ok 2019-12-09T10:31:03.454 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:18:39.718 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:31:03.478 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:18:39.742 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:31:03.481 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:18:39.704 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:31:03.487 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:18:39.710 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:31:03.487 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:18:39.711 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:31:03.493 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:18:39.716 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:31:03.599 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:18:39.822 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:31:03.631 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:18:39.854 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:31:03.653 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:18:39.918 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:14.676 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:23:50.940 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:14.689 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:23:50.913 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:14.693 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:23:50.917 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:14.696 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:23:50.919 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:14.697 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:23:50.921 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:14.703 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:23:50.968 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:14.893 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:23:51.116 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:14.916 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:23:51.139 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:14.926 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:23:51.191 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:54.759 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:24:31.024 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:54.769 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:24:31.033 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:54.786 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:24:31.009 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:54.796 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:24:31.020 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:54.808 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:24:31.072 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:54.808 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:24:31.031 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:54.813 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:24:31.036 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:54.833 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:24:31.056 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:54.858 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:24:31.082 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:26.955 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:29:03.219 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:26.964 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:29:03.228 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:26.975 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:29:03.197 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:26.975 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:29:03.198 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:26.977 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:29:03.200 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:26.978 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:29:03.201 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:27.144 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:29:03.409 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:27.166 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:29:03.389 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:27.195 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:29:03.418 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:45:16.861 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:32:53.125 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:45:16.921 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:32:53.185 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:45:16.944 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:32:53.167 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:45:16.946 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:32:53.169 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:45:16.956 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:32:53.179 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:45:16.975 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:32:53.199 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:45:17.058 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:32:53.281 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:45:17.102 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:32:53.366 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:45:17.112 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:32:53.335 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:39.038 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:34:15.302 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:39.054 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:34:15.277 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:39.058 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:34:15.322 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:39.059 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:34:15.282 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:39.071 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:34:15.294 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:39.077 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:34:15.299 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:39.133 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:34:15.356 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:39.156 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:34:15.379 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:39.204 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:34:15.468 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:48:29.465 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_skip_special ... ok 2019-12-09T10:48:34.886 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_basic ... ok 2019-12-09T10:48:40.730 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_basic ... ok 2019-12-09T10:48:47.625 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_alt ... ok 2019-12-09T10:48:55.203 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_alt ... ok 2019-12-09T10:49:01.858 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_empty ... ok 2019-12-09T10:49:10.866 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_empty ... ok 2019-12-09T10:49:17.110 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_none ... ok 2019-12-09T10:49:25.839 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_none ... ok 2019-12-09T10:49:30.664 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_not_exist ... ok 2019-12-09T10:49:37.534 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_not_exist ... ok 2019-12-09T10:49:45.557 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_unreadable ... ok 2019-12-09T10:49:50.814 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_unreadable ... ok 2019-12-09T10:49:59.186 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_basic ... ok 2019-12-09T10:50:05.773 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_basic ... ok 2019-12-09T10:50:14.225 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_alt ... ok 2019-12-09T10:50:20.941 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_alt ... ok 2019-12-09T10:50:27.633 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_not_exist ... ok 2019-12-09T10:50:36.161 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_not_exist ... ok 2019-12-09T10:50:46.914 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_delimiter_not_exist ... ok 2019-12-09T10:50:54.421 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_delimiter_not_exist ... ok 2019-12-09T10:51:03.864 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist ... ok 2019-12-09T10:51:11.993 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist ... ok 2019-12-09T10:51:21.862 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_one ... ok 2019-12-09T10:51:29.553 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_one ... ok 2019-12-09T10:51:37.333 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_zero ... ok 2019-12-09T10:51:47.002 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_zero ... ok 2019-12-09T10:51:51.541 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:39:27.804 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:51.550 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:39:27.814 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:51.560 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:39:27.783 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:51.561 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:39:27.784 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:51.563 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:39:27.787 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:51.565 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:39:27.788 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:51.753 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:39:28.016 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:51.791 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:39:28.014 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:51.795 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:39:28.018 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:52.410 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_none ... ok 2019-12-09T10:51:58.656 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_none ... ok 2019-12-09T10:52:37.101 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_unordered ... ok 2019-12-09T10:53:14.755 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_unordered ... ok 2019-12-09T10:53:21.770 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_invalid ... ok 2019-12-09T10:53:29.253 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_none ... ok 2019-12-09T10:53:36.722 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_empty ... ok 2019-12-09T10:53:38.858 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:41:15.122 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:38.915 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:41:15.138 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:38.916 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:41:15.140 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:38.923 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:41:15.146 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:38.933 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:41:15.156 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:38.938 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:41:15.202 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:38.990 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:41:15.213 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:39.033 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:41:15.256 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:39.158 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:41:15.421 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:44.997 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_continuationtoken_empty ... ok 2019-12-09T10:53:54.009 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_unreadable ... ok 2019-12-09T10:54:03.105 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_unreadable ... ok 2019-12-09T10:54:10.344 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_not_in_list ... ok 2019-12-09T10:54:17.302 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_not_in_list ... ok 2019-12-09T10:54:25.682 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_after_list ... ok 2019-12-09T10:54:33.710 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_after_list ... ok 2019-12-09T10:54:41.528 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data ... ok 2019-12-09T10:54:59.172 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data_versioning ... ok 2019-12-09T10:55:03.659 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous ... ok 2019-12-09T10:55:08.360 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous ... ok 2019-12-09T10:55:10.963 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous_fail ... ok 2019-12-09T10:55:13.188 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous_fail ... ok 2019-12-09T10:55:13.430 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_notexist ... ok 2019-12-09T10:55:13.714 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_notexist ... ok 2019-12-09T10:55:13.895 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_notexist ... ok 2019-12-09T10:55:14.651 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:42:50.914 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:55:14.671 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:42:50.934 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:55:14.719 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:42:50.942 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:55:14.733 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:42:50.956 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:55:14.758 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:42:50.981 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:55:14.761 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:42:50.984 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:55:14.792 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:42:51.056 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:55:14.891 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:42:51.114 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:55:14.897 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:42:51.120 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:55:17.775 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_nonempty ... ok 2019-12-09T10:55:23.701 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_concurrent_set_canned_acl ... ok 2019-12-09T10:55:23.887 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_write_to_nonexist_bucket ... ok 2019-12-09T10:55:30.026 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_delete ... ok 2019-12-09T10:55:33.688 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_read_notexist ... ok 2019-12-09T10:55:35.614 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_requestid_matches_header_on_error ... ok 2019-12-09T10:55:46.497 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multi_object_delete ... ok 2019-12-09T10:55:56.790 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multi_objectv2_delete ... ok 2019-12-09T10:55:59.969 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_head_zero_bytes ... ok 2019-12-09T10:56:04.203 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_write_check_etag ... ok 2019-12-09T10:56:07.317 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_write_cache_control ... ok 2019-12-09T10:56:10.968 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_write_expires ... ok 2019-12-09T10:56:19.361 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_write_read_update_read_delete ... ok 2019-12-09T10:56:23.625 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_good ... ok 2019-12-09T10:56:28.125 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_empty ... ok 2019-12-09T10:56:35.852 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_overwrite_to_empty ... ok 2019-12-09T10:56:39.371 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_unicode_metadata ... ok 2019-12-09T10:56:46.853 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_metadata_replaced_on_put ... ok 2019-12-09T10:56:52.160 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_write_file ... ok 2019-12-09T10:56:57.006 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_anonymous_request ... ok 2019-12-09T10:57:01.046 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request ... ok 2019-12-09T10:57:02.182 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:44:38.445 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:57:02.197 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:44:38.460 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:57:02.201 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:44:38.424 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:57:02.206 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:44:38.429 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:57:02.213 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:44:38.436 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:57:02.221 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:44:38.444 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:57:02.299 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:44:38.522 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T10:57:02.312 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:44:38.573 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T10:57:02.342 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:44:38.565 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T10:57:05.815 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_no_content_type ... ok 2019-12-09T10:57:11.063 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request_bad_access_key ... ok 2019-12-09T10:57:15.118 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_success_code ... ok 2019-12-09T10:57:19.295 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_invalid_success_code ... ok 2019-12-09T10:57:25.468 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_larger_than_chunk ... ok 2019-12-09T10:57:29.517 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_key_from_filename ... ok 2019-12-09T10:57:35.259 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_ignored_header ... ok 2019-12-09T10:57:40.062 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_case_insensitive_condition_fields ... ok 2019-12-09T10:57:44.414 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_escaped_field_values ... ok 2019-12-09T10:57:50.568 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_success_redirect_action ... ok 2019-12-09T10:57:53.521 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_signature ... ok 2019-12-09T10:57:56.436 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_access_key ... ok 2019-12-09T10:57:59.427 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_date_format ... ok 2019-12-09T10:58:02.059 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_no_key_specified ... ok 2019-12-09T10:58:04.600 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_signature ... ok 2019-12-09T10:58:08.389 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_policy_condition ... ok 2019-12-09T10:58:12.686 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_user_specified_header ... ok 2019-12-09T10:58:15.382 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_request_missing_policy_specified_field ... ok 2019-12-09T10:58:17.659 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_condition_is_case_sensitive ... ok 2019-12-09T10:58:20.314 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_expires_is_case_sensitive ... ok 2019-12-09T10:58:23.425 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_expired_policy ... ok 2019-12-09T10:58:27.092 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_request_field_value ... ok 2019-12-09T10:58:31.097 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_expires_condition ... ok 2019-12-09T10:58:33.681 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_conditions_list ... ok 2019-12-09T10:58:37.105 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_limit_exceeded ... ok 2019-12-09T10:58:40.200 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_content_length_argument ... ok 2019-12-09T10:58:43.465 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_content_length_argument ... ok 2019-12-09T10:58:47.382 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_below_minimum ... ok 2019-12-09T10:58:49.529 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_empty_conditions ... ok 2019-12-09T10:58:52.908 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_good ... ok 2019-12-09T10:58:59.523 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_failed ... ok 2019-12-09T10:59:03.414 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_good ... ok 2019-12-09T10:59:07.878 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_failed ... ok 2019-12-09T10:59:13.746 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_good ... ok 2019-12-09T10:59:20.447 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_failed ... ok 2019-12-09T10:59:24.319 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_good ... ok 2019-12-09T10:59:29.881 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_failed ... ok 2019-12-09T10:59:35.813 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_good ... ok 2019-12-09T10:59:41.772 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_failed ... ok 2019-12-09T10:59:49.214 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_overwrite_existed_good ... ok 2019-12-09T10:59:52.406 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_nonexisted_failed ... ok 2019-12-09T10:59:59.505 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_good ... ok 2019-12-09T11:00:04.762 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_failed ... ok 2019-12-09T11:00:09.341 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_nonexisted_good ... ok 2019-12-09T11:00:16.030 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_overwrite_existed_failed ... ok 2019-12-09T11:00:22.976 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get ... ok 2019-12-09T11:00:32.813 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_gone ... ok 2019-12-09T11:00:43.262 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_delete_key_bucket_gone ... ok 2019-12-09T11:00:49.548 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_gone ... ok 2019-12-09T11:00:52.176 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_head ... ok 2019-12-09T11:01:02.036 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_head_extended ... ok 2019-12-09T11:01:06.607 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_acl ... ok 2019-12-09T11:01:11.939 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_acl ... ok 2019-12-09T11:01:15.488 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated ... ok 2019-12-09T11:01:19.874 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_response_headers ... ok 2019-12-09T11:01:23.559 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_acl ... ok 2019-12-09T11:01:27.402 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_acl ... ok 2019-12-09T11:01:36.568 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_gone ... ok 2019-12-09T11:01:42.112 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_gone ... ok 2019-12-09T11:01:45.636 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_not_expired ... ok 2019-12-09T11:01:49.341 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_range_zero ... ok 2019-12-09T11:01:54.521 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_max_range ... ok 2019-12-09T11:01:59.241 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_positive_range ... ok 2019-12-09T11:02:01.204 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:49:37.466 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:01.212 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:49:37.474 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:01.253 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:49:37.475 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:01.254 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:49:37.477 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:01.262 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:49:37.485 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:01.271 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:49:37.494 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:01.446 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:49:37.709 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:01.470 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:49:37.693 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:01.471 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:49:37.694 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:02.444 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put ... ok 2019-12-09T11:02:08.372 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put_write_access ... ok 2019-12-09T11:02:12.366 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:49:48.629 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:12.373 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:49:48.636 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:12.377 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:49:48.600 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:12.382 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:49:48.605 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:12.385 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:49:48.609 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:12.391 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:49:48.614 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:12.406 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:49:48.628 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:12.421 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:49:48.644 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:12.447 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:49:48.710 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:12.608 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_put_authenticated ... ok 2019-12-09T11:02:16.273 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_raw_put_authenticated_expired ... ok 2019-12-09T11:02:16.650 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_one ... ok 2019-12-09T11:02:17.399 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_two ... ok 2019-12-09T11:02:17.989 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_ip ... ok 2019-12-09T11:02:18.193 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_underscore ... ok 2019-12-09T11:02:25.331 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists ... ok 2019-12-09T11:02:28.568 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_get_location ... ok 2019-12-09T11:02:32.538 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists_nonowner ... ok 2019-12-09T11:02:35.635 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_default ... ok 2019-12-09T11:02:39.282 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_during_create ... ok 2019-12-09T11:02:43.142 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned ... ok 2019-12-09T11:02:46.006 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_publicreadwrite ... ok 2019-12-09T11:02:48.351 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_authenticatedread ... ok 2019-12-09T11:02:52.495 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_default ... ok 2019-12-09T11:02:57.625 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_during_create ... ok 2019-12-09T11:03:03.701 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned ... ok 2019-12-09T11:03:07.401 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_publicreadwrite ... ok 2019-12-09T11:03:11.423 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_authenticatedread ... ok 2019-12-09T11:03:18.329 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerread ... ok 2019-12-09T11:03:24.467 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerfullcontrol ... ok 2019-12-09T11:03:32.376 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_owner ... ok 2019-12-09T11:03:37.966 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_attributes ... ok 2019-12-09T11:03:41.885 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_private_to_private ... ok 2019-12-09T11:03:47.977 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl ... ok 2019-12-09T11:03:53.959 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_write ... ok 2019-12-09T11:04:00.201 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_writeacp ... ok 2019-12-09T11:04:04.894 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_read ... ok 2019-12-09T11:04:11.151 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_acl_readacp ... ok 2019-12-09T11:04:20.031 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_fullcontrol ... ok 2019-12-09T11:04:23.950 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_read ... ok 2019-12-09T11:04:29.339 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_readacp ... ok 2019-12-09T11:04:33.326 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_write ... ok 2019-12-09T11:04:37.973 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_writeacp ... ok 2019-12-09T11:04:40.190 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_nonexist_user ... ok 2019-12-09T11:04:46.590 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_no_grants ... ok 2019-12-09T11:04:50.082 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_header_acl_grants ... ok 2019-12-09T11:04:55.137 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_header_acl_grants ... ok 2019-12-09T11:04:57.994 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email ... ok 2019-12-09T11:05:00.136 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email_notexist ... ok 2019-12-09T11:05:06.592 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_revoke_all ... ok 2019-12-09T11:05:13.633 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_private ... ok 2019-12-09T11:05:21.661 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_private ... ok 2019-12-09T11:05:29.438 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicread ... ok 2019-12-09T11:05:38.699 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicread ... ok 2019-12-09T11:05:45.577 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicreadwrite ... ok 2019-12-09T11:05:52.361 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicreadwrite ... ok 2019-12-09T11:06:00.423 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_private ... ok 2019-12-09T11:06:09.127 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicread ... ok 2019-12-09T11:06:17.904 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicreadwrite ... ok 2019-12-09T11:06:34.831 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_private ... ok 2019-12-09T11:06:48.049 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicread ... ok 2019-12-09T11:07:02.682 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicreadwrite ... ok 2019-12-09T11:07:18.517 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_buckets_create_then_list ... ok 2019-12-09T11:07:19.270 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_anonymous ... ok 2019-12-09T11:07:19.530 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_invalid_auth ... ok 2019-12-09T11:07:19.565 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_bad_auth ... ok 2019-12-09T11:07:25.854 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:55:02.115 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:25.858 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:55:02.120 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:25.874 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:55:02.097 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:25.878 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:55:02.101 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:25.879 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:55:02.102 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:25.884 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:55:02.107 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:26.076 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:55:02.338 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:26.077 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:55:02.300 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:26.088 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:55:02.311 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:29.613 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_alpha ... ok 2019-12-09T11:07:39.780 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_digit ... ok 2019-12-09T11:07:43.339 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_period ... ok 2019-12-09T11:07:45.895 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_hyphen ... ok 2019-12-09T11:07:53.849 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_recreate_not_overriding ... ok 2019-12-09T11:08:29.282 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_special_key_names ... ok 2019-12-09T11:08:41.918 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_special_prefix ... ok 2019-12-09T11:08:50.768 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_zero_size ... ok 2019-12-09T11:08:57.094 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_same_bucket ... ok 2019-12-09T11:09:02.939 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_verify_contenttype ... ok 2019-12-09T11:09:07.988 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself ... ok 2019-12-09T11:09:13.943 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself_with_metadata ... ok 2019-12-09T11:09:21.290 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_diff_bucket ... ok 2019-12-09T11:09:29.548 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_bucket ... ok 2019-12-09T11:09:40.073 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_object_bucket ... ok 2019-12-09T11:09:47.489 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_canned_acl ... ok 2019-12-09T11:10:01.512 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_retaining_metadata ... ok 2019-12-09T11:10:15.381 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_replacing_metadata ... ok 2019-12-09T11:10:17.613 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_bucket_not_found ... ok 2019-12-09T11:10:20.427 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_key_not_found ... ok 2019-12-09T11:10:23.408 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:57:59.670 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:23.417 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:57:59.679 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:23.443 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:57:59.666 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:23.462 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:57:59.685 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:23.488 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:57:59.711 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:23.493 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:57:59.716 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:23.531 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:01:57:59.793 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:23.574 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:01:57:59.797 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:23.589 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:01:57:59.812 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:56.758 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_bucket ... ok 2019-12-09T11:11:09.894 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_url_encoding ... ok 2019-12-09T11:11:53.751 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:11:53.626+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:12:33.644 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioning_multipart_upload ... ok 2019-12-09T11:12:37.747 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_empty ... ok 2019-12-09T11:12:38.071 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:00:14.333 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:38.095 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:00:14.318 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:38.104 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:00:14.367 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:38.107 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:00:14.330 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:38.108 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:00:14.331 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:38.115 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:00:14.339 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:38.229 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:00:14.452 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:38.239 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:00:14.462 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:38.282 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:00:14.544 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:45.978 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_small ... ok 2019-12-09T11:13:06.983 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:13:06.955+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:13:07.097 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_small ... ok 2019-12-09T11:13:13.026 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_invalid_range ... ok 2019-12-09T11:13:29.646 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_without_range ... ok 2019-12-09T11:13:49.930 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:13:49.868+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:14:07.096 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:14:07.090+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:14:35.769 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_special_names ... ok 2019-12-09T11:15:04.893 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:15:04.853+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:15:34.821 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:15:34.755+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:15:37.695 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload ... ok 2019-12-09T11:16:22.184 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:16:22.100+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:16:41.117 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:16:41.096+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:16:44.535 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:16:44.386+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:17:01.825 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:17:01.675+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:17:05.093 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:17:05.087+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:17:06.273 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_versioned ... ok 2019-12-09T11:17:06.389 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:17:06.359+0000 40a63700 -1 handle_data failed with exit code -5 2019-12-09T11:17:50.196 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:05:26.458 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:50.229 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:05:26.492 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:50.461 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:05:26.723 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:50.549 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:05:26.772 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:50.557 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:05:26.780 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:50.559 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:05:26.782 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:50.565 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:05:26.788 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:50.777 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:05:27.000 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:50.798 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:05:27.021 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:45.980 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:06:22.243 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:45.990 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:06:22.253 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.038 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:06:22.261 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.041 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:06:22.263 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.047 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:06:22.270 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.069 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:06:22.292 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.102 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:06:22.364 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.103 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:06:22.326 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.124 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:06:22.347 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:20:50.298 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:20:50.213+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:21:29.000 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:21:28.929+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:23:00.511 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:10:36.773 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:23:00.524 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:10:36.787 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:23:00.531 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:10:36.754 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:23:00.532 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:10:36.755 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:23:00.539 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:10:36.762 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:23:00.539 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:10:36.762 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:23:00.713 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:10:36.975 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:23:00.737 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:10:36.959 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:23:00.748 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:10:36.971 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:03.139 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:24:03.113+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:24:31.042 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:24:31.027+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:24:33.526 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_resend_part ... ok 2019-12-09T11:25:51.994 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_multiple_sizes ... ok 2019-12-09T11:26:06.809 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:26:06.766+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:26:20.079 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:26:19.983+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:26:36.165 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:26:36.078+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:26:54.451 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:26:54.401+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:27:08.084 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:14:44.347 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:08.160 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:14:44.424 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:08.193 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:14:44.416 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:08.197 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:14:44.420 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:08.221 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:14:44.443 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:08.226 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:14:44.448 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:08.353 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:14:44.576 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:08.362 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:14:44.625 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:08.416 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:14:44.639 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:13.141 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:27:12.631+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:27:27.299 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T11:27:27.245+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:27:28.455 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_multiple_sizes ... ok 2019-12-09T11:27:57.185 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_size_too_small ... ok 2019-12-09T11:28:11.562 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:15:47.825 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:28:11.566 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:15:47.829 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:28:11.578 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:15:47.801 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:28:11.581 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:15:47.804 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:28:11.582 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:15:47.805 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:28:11.591 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:15:47.814 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:28:11.649 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:15:47.872 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:28:11.650 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:15:47.914 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:28:11.665 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:15:47.888 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:28:18.312 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_contents ... ok 2019-12-09T11:28:40.487 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_overwrite_existing_object ... ok 2019-12-09T11:28:54.146 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload ... ok 2019-12-09T11:28:57.653 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload_not_found ... ok 2019-12-09T11:29:27.635 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_list_multipart_upload ... ok 2019-12-09T11:29:36.630 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_missing_part ... ok 2019-12-09T11:29:46.342 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_incorrect_etag ... ok 2019-12-09T11:29:49.709 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_100_continue ... ok 2019-12-09T11:29:56.565 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_set_cors ... ok 2019-12-09T11:30:20.444 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_response ... ok 2019-12-09T11:30:29.655 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_wildcard ... ok 2019-12-09T11:30:37.413 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_cors_header_option ... ok 2019-12-09T11:30:48.156 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_1mb ... ok 2019-12-09T11:30:59.555 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_4mb ... ok 2019-12-09T11:31:12.106 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_8mb ... ok 2019-12-09T11:31:24.912 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_1mb ... ok 2019-12-09T11:31:38.779 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_4mb ... ok 2019-12-09T11:31:54.196 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_8mb ... ok 2019-12-09T11:32:09.946 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_1mb ... ok 2019-12-09T11:32:29.873 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_4mb ... ok 2019-12-09T11:32:53.312 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_8mb ... ok 2019-12-09T11:33:02.103 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_atomic_conditional_write_1mb ... ok 2019-12-09T11:33:11.022 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_atomic_multipart_upload_write ... ok 2019-12-09T11:33:22.985 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:20:59.248 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:22.999 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:20:59.262 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:23.026 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:20:59.248 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:23.026 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:20:59.249 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:23.032 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:20:59.255 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:23.035 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:20:59.258 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:23.249 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:20:59.512 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:23.303 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:20:59.526 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:23.359 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:20:59.582 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:27.939 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_multipart_resend_first_finishes_last ... ok 2019-12-09T11:33:32.716 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_response_code ... ok 2019-12-09T11:33:44.270 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_ranged_big_request_response_code ... ok 2019-12-09T11:33:48.454 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_skip_leading_bytes_response_code ... ok 2019-12-09T11:33:52.926 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_return_trailing_bytes_response_code ... ok 2019-12-09T11:33:57.900 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_invalid_range ... ok 2019-12-09T11:34:00.726 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_empty_object ... ok 2019-12-09T11:34:08.312 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_create_suspend ... ok 2019-12-09T11:35:30.227 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:23:06.490 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:30.236 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:23:06.499 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:30.281 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:23:06.504 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:30.293 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:23:06.516 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:30.296 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:23:06.519 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:30.299 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:23:06.522 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:30.340 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:23:06.603 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:30.386 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:23:06.609 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:30.399 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:23:06.622 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:55.580 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:23:31.843 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:55.621 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:23:31.884 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:55.639 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:23:31.862 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:55.644 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:23:31.867 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:55.657 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:23:31.880 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:55.660 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:23:31.883 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:55.660 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:23:31.924 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:55.695 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:23:31.918 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:35:55.713 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:23:31.936 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.187 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:26:08.450 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.213 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:26:08.416 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.213 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:26:08.421 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.213 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:26:08.418 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.213 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:26:08.422 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.213 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:26:08.461 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.311 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:26:08.533 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.317 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:26:08.580 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.353 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:26:08.576 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:39:29.751 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove ... ok 2019-12-09T11:40:29.940 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove_head ... ok 2019-12-09T11:40:42.348 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_removal ... ok 2019-12-09T11:41:01.649 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite ... ok 2019-12-09T11:41:18.490 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite_suspended ... ok 2019-12-09T11:43:27.936 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_suspend_versions ... ok 2019-12-09T11:43:43.705 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:31:19.968 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:43.722 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:31:19.984 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:43.731 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:31:19.953 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:43.735 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:31:19.958 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:43.738 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:31:19.961 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:43.742 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:31:19.965 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:43.945 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:31:20.207 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:43.954 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:31:20.177 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:44.075 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:31:20.298 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:52.376 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:31:28.639 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:52.382 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:31:28.645 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:52.403 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:31:28.626 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:52.404 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:31:28.667 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:52.406 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:31:28.629 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:52.421 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:31:28.644 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:52.424 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:31:28.647 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:52.444 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:31:28.667 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:52.449 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:31:28.672 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:45:14.439 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_all ... ok 2019-12-09T11:48:55.932 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:36:32.194 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:55.950 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:36:32.173 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:55.950 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:36:32.198 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:55.964 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:36:32.184 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:55.964 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:36:32.186 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:55.966 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:36:32.188 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:56.187 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:36:32.450 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:56.249 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:36:32.471 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:56.257 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:36:32.480 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:52:19.578 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:39:55.841 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:52:19.605 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:39:55.867 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:52:19.665 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:39:55.887 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:52:19.676 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:39:55.898 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:52:19.680 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:39:55.903 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:52:19.708 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:39:55.931 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:52:19.785 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:39:56.008 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:52:19.795 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:39:56.018 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:52:19.802 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:39:56.065 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:52:57.614 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_special_names ... ok 2019-12-09T11:54:10.159 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:41:46.422 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:54:10.184 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:41:46.396 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:54:10.185 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:41:46.400 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:54:10.185 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:41:46.402 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:54:10.185 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:41:46.407 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:54:10.190 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:41:46.453 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:54:10.292 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:41:46.515 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:54:10.310 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:41:46.533 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:54:10.351 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:41:46.614 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:54:27.974 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_overwrite_multipart ... ok 2019-12-09T11:55:03.432 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_list_marker ... ok 2019-12-09T11:55:36.275 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_copy_obj_version ... ok 2019-12-09T11:55:56.626 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete ... ok 2019-12-09T11:56:28.076 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker ... ok 2019-12-09T11:56:35.881 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker_create ... ok 2019-12-09T11:56:58.274 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl ... ok 2019-12-09T11:57:18.689 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl_no_version_specified ... ok 2019-12-09T11:57:26.697 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_concurrent_remove ... Exception in thread Thread-53: 2019-12-09T11:57:26.697 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:26.698 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:26.698 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:26.698 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:26.698 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:26.698 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:26.698 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:26.698 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:26.699 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:26.699 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:26.699 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:26.699 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:26.699 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:26.807 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-51: 2019-12-09T11:57:26.807 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:26.808 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:26.808 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:26.808 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:26.808 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:26.808 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:26.808 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:26.808 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:26.809 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:26.809 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:26.809 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:26.809 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:26.809 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:27.417 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-54: 2019-12-09T11:57:27.417 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:27.417 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:27.417 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:27.417 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:27.418 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:27.418 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:27.418 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:27.418 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:27.418 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:27.419 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:27.419 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:27.419 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:27.419 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:27.423 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-55: 2019-12-09T11:57:27.423 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:27.423 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:27.423 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:27.423 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:27.424 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:27.424 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:27.424 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:27.424 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:27.424 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:27.424 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:27.424 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:27.425 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:27.425 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:34.947 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-65: 2019-12-09T11:57:34.947 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:34.947 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:34.947 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:34.948 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:34.948 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:34.948 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:34.948 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:34.948 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:34.948 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:34.949 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:34.949 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:34.952 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:34.952 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:35.736 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-63: 2019-12-09T11:57:35.736 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:35.737 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:35.737 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:35.737 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:35.737 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:35.737 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:35.737 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:35.737 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:35.738 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:35.738 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:35.738 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:35.738 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:35.738 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:35.739 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-64: 2019-12-09T11:57:35.739 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:35.739 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:35.739 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:35.739 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:35.740 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:35.740 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:35.740 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:35.740 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:35.740 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:35.740 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:35.740 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:35.741 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:35.741 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:35.959 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-61: 2019-12-09T11:57:35.959 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:35.959 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:35.959 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:35.960 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:35.960 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:35.960 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:35.960 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:35.960 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:35.960 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:35.960 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:35.961 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:35.961 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:35.961 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:46.917 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-75: 2019-12-09T11:57:46.917 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:46.917 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:46.917 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:46.917 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:46.917 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:46.918 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:46.918 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:46.918 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:46.918 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:46.918 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:46.918 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:46.918 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:46.919 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:47.184 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-72: 2019-12-09T11:57:47.185 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:47.185 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:47.185 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:47.185 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:47.185 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:47.185 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:47.186 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:47.186 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:47.186 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:47.186 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:47.186 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:47.186 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:47.186 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:48.037 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-74: 2019-12-09T11:57:48.037 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:48.038 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:48.038 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:48.038 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:48.038 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:48.039 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:48.039 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:48.039 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:48.039 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:48.040 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:48.040 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:48.040 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:48.040 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:48.041 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-73: 2019-12-09T11:57:48.041 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:48.041 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:48.041 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:48.042 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:48.042 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:48.042 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:48.042 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:48.043 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:48.043 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:48.043 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:48.043 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:48.044 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:48.044 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:57:59.527 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-83: 2019-12-09T11:57:59.527 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:57:59.527 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:57:59.527 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:57:59.527 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:57:59.527 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:57:59.528 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:57:59.528 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:57:59.528 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:57:59.528 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:57:59.528 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:57:59.529 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:57:59.529 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:57:59.529 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:00.511 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-82: 2019-12-09T11:58:00.511 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:00.511 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:00.511 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:00.511 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:00.512 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:00.512 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:00.512 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:00.512 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:00.512 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:00.512 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:00.512 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:00.513 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:00.513 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:00.513 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-84: 2019-12-09T11:58:00.513 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:00.513 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:00.514 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:00.514 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:00.514 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:00.514 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:00.514 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:00.514 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:00.514 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:00.514 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:00.515 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:00.515 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:00.515 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:00.597 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-85: 2019-12-09T11:58:00.598 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:00.598 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:00.598 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:00.598 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:00.598 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:00.598 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:00.599 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:00.599 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:00.599 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:00.599 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:00.599 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:00.599 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:00.599 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:10.921 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-95: 2019-12-09T11:58:10.922 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:10.922 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:10.922 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:10.922 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:10.922 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:10.922 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:10.922 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:10.923 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:10.923 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:10.923 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:10.923 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:10.923 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:10.923 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:11.727 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-92: 2019-12-09T11:58:11.727 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:11.728 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:11.728 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:11.728 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:11.728 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:11.728 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:11.728 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:11.729 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:11.729 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:11.729 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:11.729 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:11.729 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:11.729 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:23.357 INFO:teuthology.orchestra.run.smithi035.stderr:ok 2019-12-09T11:58:31.612 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_and_remove ... Exception in thread Thread-101: 2019-12-09T11:58:31.612 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:31.613 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:31.613 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:31.613 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:31.613 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:31.613 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:31.613 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:31.613 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:31.614 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:31.614 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:31.614 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:31.614 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:31.614 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:31.614 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-106: 2019-12-09T11:58:31.614 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:31.615 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:31.615 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:31.615 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:31.615 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:31.615 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:31.615 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:31.615 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:31.615 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:31.616 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:31.616 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:31.616 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:31.616 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:31.656 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-104: 2019-12-09T11:58:31.656 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:31.656 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:31.656 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:31.657 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:31.657 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:31.657 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:31.657 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:31.657 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:31.657 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:31.657 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:31.657 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:31.658 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:31.658 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:31.969 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-102: 2019-12-09T11:58:31.969 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:31.969 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:31.969 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:31.969 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:31.970 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:31.970 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:31.970 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:31.970 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:31.970 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:31.970 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:31.970 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:31.971 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:31.971 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:32.081 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-107: 2019-12-09T11:58:32.081 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:32.082 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:32.082 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:32.082 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:32.082 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:32.082 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:32.082 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:32.082 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:32.083 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:32.083 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:32.083 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:32.083 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:32.083 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:32.116 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-105: 2019-12-09T11:58:32.116 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:32.116 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:32.116 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:32.117 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:32.117 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:32.117 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:32.117 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:32.117 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:32.117 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:32.117 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:32.118 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:32.118 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:32.118 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:32.129 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-108: 2019-12-09T11:58:32.129 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:32.129 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:32.129 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:32.129 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:32.129 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:32.130 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:32.130 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:32.130 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:32.130 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:32.130 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:32.130 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:32.131 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:32.131 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:32.160 INFO:teuthology.orchestra.run.smithi035.stderr:Exception in thread Thread-109: 2019-12-09T11:58:32.161 INFO:teuthology.orchestra.run.smithi035.stderr:Traceback (most recent call last): 2019-12-09T11:58:32.161 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T11:58:32.161 INFO:teuthology.orchestra.run.smithi035.stderr: self.run() 2019-12-09T11:58:32.161 INFO:teuthology.orchestra.run.smithi035.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T11:58:32.161 INFO:teuthology.orchestra.run.smithi035.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T11:58:32.161 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T11:58:32.161 INFO:teuthology.orchestra.run.smithi035.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T11:58:32.162 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T11:58:32.162 INFO:teuthology.orchestra.run.smithi035.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T11:58:32.162 INFO:teuthology.orchestra.run.smithi035.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T11:58:32.162 INFO:teuthology.orchestra.run.smithi035.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T11:58:32.162 INFO:teuthology.orchestra.run.smithi035.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T11:58:32.162 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T11:58:44.903 INFO:teuthology.orchestra.run.smithi035.stderr:ok 2019-12-09T11:58:48.256 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set ... ok 2019-12-09T11:58:52.523 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get ... ok 2019-12-09T11:58:57.622 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get_no_id ... ok 2019-12-09T11:59:00.329 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_id_too_long ... ok 2019-12-09T11:59:04.028 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_same_id ... ok 2019-12-09T11:59:05.861 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_invalid_status ... ok 2019-12-09T11:59:09.935 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_date ... ok 2019-12-09T11:59:12.613 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_invalid_date ... ok 2019-12-09T11:59:21.026 INFO:teuthology.orchestra.run.smithi035.stderr:Check for valid x-amz-expiration header after PUT ... ok 2019-12-09T11:59:21.503 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:46:57.766 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:21.523 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:46:57.746 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:21.524 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:46:57.788 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:21.530 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:46:57.753 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:21.539 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:46:57.762 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:21.550 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:46:57.773 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:21.773 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:46:58.036 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:21.820 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:46:58.043 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:21.896 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:46:58.119 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:24.688 INFO:teuthology.orchestra.run.smithi035.stderr:Check for valid x-amz-expiration header on HEAD request ... ok 2019-12-09T11:59:31.655 INFO:teuthology.orchestra.run.smithi035.stderr:Check for valid x-amz-expiration header on GET request ... ok 2019-12-09T11:59:41.905 INFO:teuthology.orchestra.run.smithi035.stderr:Check x-amz-expiration-header object tags interaction ... ok 2019-12-09T11:59:49.260 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_noncurrent ... ok 2019-12-09T11:59:53.696 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_deletemarker ... ok 2019-12-09T11:59:56.773 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_filter ... ok 2019-12-09T12:00:00.804 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_empty_filter ... ok 2019-12-09T12:00:11.447 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_multipart ... ok 2019-12-09T12:00:15.794 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1b ... ok 2019-12-09T12:00:18.975 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1kb ... ok 2019-12-09T12:00:24.625 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1MB ... ok 2019-12-09T12:00:28.717 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_13b ... ok 2019-12-09T12:00:34.183 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_method_head ... ok 2019-12-09T12:00:38.925 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_present ... ok 2019-12-09T12:00:41.656 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:48:17.919 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:41.666 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:48:17.929 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:41.705 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:48:17.928 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:41.706 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:48:17.929 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:41.708 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:48:17.931 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:41.709 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:48:17.932 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:41.764 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:48:17.987 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:41.770 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:48:18.033 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:41.809 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:48:18.032 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:42.233 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_other_key ... ok 2019-12-09T12:00:43.646 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_invalid_md5 ... ok 2019-12-09T12:00:46.415 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_md5 ... ok 2019-12-09T12:00:50.856 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_key ... ok 2019-12-09T12:00:54.292 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encryption_key_no_sse_c ... ok 2019-12-09T12:02:24.905 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_upload ... ok 2019-12-09T12:03:03.115 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_bad_download ... ok 2019-12-09T12:03:06.396 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_post_object_authenticated_request ... ok 2019-12-09T12:03:10.358 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_method_head ... ok 2019-12-09T12:03:14.450 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_present ... ok 2019-12-09T12:03:17.851 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_no_key ... ok 2019-12-09T12:03:20.640 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_not_declared ... ok 2019-12-09T12:04:30.572 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:52:06.836 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:30.591 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:52:06.814 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:30.592 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:52:06.815 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:30.593 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:52:06.857 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:30.596 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:52:06.819 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:30.598 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:52:06.821 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:30.767 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:52:06.988 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:30.814 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:52:07.034 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:30.816 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:52:07.076 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:42.798 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_upload ... ok 2019-12-09T12:05:10.542 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_1 ... ok 2019-12-09T12:05:42.319 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_2 ... ok 2019-12-09T12:05:45.091 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_post_object_authenticated_request ... ok 2019-12-09T12:05:48.334 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1b ... ok 2019-12-09T12:05:51.854 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1kb ... ok 2019-12-09T12:05:56.051 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1MB ... ok 2019-12-09T12:05:59.197 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_13b ... ok 2019-12-09T12:06:01.490 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_read_declare ... ok 2019-12-09T12:06:06.402 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy ... ok 2019-12-09T12:06:10.892 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy ... ok 2019-12-09T12:06:16.587 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_acl ... ok 2019-12-09T12:06:22.550 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_acl ... ok 2019-12-09T12:06:31.629 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_another_bucket ... ok 2019-12-09T12:06:42.875 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_another_bucket ... ok 2019-12-09T12:06:49.945 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_get_obj_tagging ... ok 2019-12-09T12:06:55.893 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_get_obj_head_tagging ... ok 2019-12-09T12:07:05.481 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_max_tags ... ok 2019-12-09T12:07:10.612 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_excess_tags ... ok 2019-12-09T12:07:17.969 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_max_kvsize_tags ... ok 2019-12-09T12:07:25.087 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_excess_key_tags ... ok 2019-12-09T12:07:31.325 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_excess_val_tags ... ok 2019-12-09T12:07:38.726 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_modify_tags ... ok 2019-12-09T12:07:48.837 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_delete_tags ... ok 2019-12-09T12:07:53.053 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_anonymous_request ... ok 2019-12-09T12:07:57.181 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_authenticated_request ... ok 2019-12-09T12:08:03.064 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_obj_with_tags ... ok 2019-12-09T12:08:09.928 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_get_tags_acl_public ... ok 2019-12-09T12:08:17.871 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_put_tags_acl_public ... ok 2019-12-09T12:08:30.737 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_delete_tags_obj_public ... ok 2019-12-09T12:08:48.654 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_atomic_upload_return_version_id ... ok 2019-12-09T12:09:03.800 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:56:40.064 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:03.810 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:56:40.073 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:03.879 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:56:40.102 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:03.901 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:56:40.124 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:03.907 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:56:40.130 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:03.917 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:56:40.140 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:03.953 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:56:40.216 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:04.058 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:56:40.281 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:04.074 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:56:40.296 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:41.518 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:57:17.782 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:41.528 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:57:17.792 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:41.538 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:57:17.760 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:41.538 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:57:17.761 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:41.541 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:57:17.764 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:41.542 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:57:17.765 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:41.590 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:02:57:17.813 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:41.592 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:02:57:17.815 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:41.635 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:02:57:17.899 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:10:24.056 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_multipart_upload_return_version_id ... ok 2019-12-09T12:10:36.476 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_existing_tag ... ok 2019-12-09T12:10:51.812 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_tagging_existing_tag ... ok 2019-12-09T12:11:03.728 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_tagging_existing_tag ... ok 2019-12-09T12:11:18.173 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source ... ok 2019-12-09T12:11:28.554 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source_meta ... ok 2019-12-09T12:11:34.806 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_acl ... ok 2019-12-09T12:11:47.648 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_grant ... ok 2019-12-09T12:11:58.426 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_acl_existing_tag ... ok 2019-12-09T12:12:03.288 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock ... ok 2019-12-09T12:12:07.174 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_bucket ... ok 2019-12-09T12:12:09.257 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_with_days_and_years ... ok 2019-12-09T12:12:11.174 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_days ... ok 2019-12-09T12:12:14.516 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_years ... ok 2019-12-09T12:12:16.965 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_status ... ok 2019-12-09T12:12:19.654 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_suspend_versioning ... ok 2019-12-09T12:12:24.044 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock ... ok 2019-12-09T12:12:26.272 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock_invalid_bucket ... ok 2019-12-09T12:12:38.732 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention ... ok 2019-12-09T12:12:41.535 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_bucket ... ok 2019-12-09T12:12:48.059 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_mode ... ok 2019-12-09T12:13:01.861 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention ... ok 2019-12-09T12:13:07.047 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention_invalid_bucket ... ok 2019-12-09T12:13:20.845 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_versionid ... ok 2019-12-09T12:13:34.175 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_override_default_retention ... ok 2019-12-09T12:13:50.887 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_increase_period ... ok 2019-12-09T12:14:03.969 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period ... ok 2019-12-09T12:14:17.215 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period_bypass ... ok 2019-12-09T12:14:27.621 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_retention ... ok 2019-12-09T12:14:34.446 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold ... ok 2019-12-09T12:14:37.459 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_bucket ... ok 2019-12-09T12:14:41.170 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_status ... ok 2019-12-09T12:14:48.300 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold ... ok 2019-12-09T12:14:51.522 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold_invalid_bucket ... ok 2019-12-09T12:14:52.912 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:02:29.176 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:52.928 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:02:29.151 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:52.932 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:02:29.155 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:52.932 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:02:29.155 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:52.937 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:02:29.160 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:52.943 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:02:29.207 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:53.124 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:02:29.347 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:53.132 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:02:29.354 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:53.217 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:02:29.481 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:57.244 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_on ... ok 2019-12-09T12:15:05.813 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_off ... ok 2019-12-09T12:15:18.089 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_metadata ... ok 2019-12-09T12:15:25.579 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_s3.test_object_lock_uploading_obj ... ok 2019-12-09T12:15:25.605 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.functional.test_utils.test_generate ... ok 2019-12-09T12:16:10.816 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:03:47.079 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:10.827 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:03:47.091 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:10.905 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:03:47.161 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:10.941 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:03:47.164 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:10.947 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:03:47.170 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:10.971 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:03:47.194 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:10.974 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:03:47.197 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:11.019 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:03:47.242 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:11.060 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:03:47.282 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:17:26.143 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:05:02.407 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:17:26.159 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:05:02.423 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:17:26.225 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:05:02.447 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:17:26.227 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:05:02.450 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:17:26.229 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:05:02.452 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:17:26.242 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:05:02.465 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:17:26.255 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:05:02.520 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:17:26.275 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:05:02.497 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:17:26.306 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:05:02.529 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:01.893 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:07:38.157 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:01.910 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:07:38.168 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:01.916 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:07:38.139 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:01.926 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:07:38.149 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:01.957 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:07:38.179 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:01.960 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:07:38.182 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:02.025 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:07:38.289 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:02.068 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:07:38.290 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:02.124 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:07:38.346 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:15.350 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:12:51.614 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:15.357 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:12:51.621 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:15.362 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:12:51.585 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:15.364 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:12:51.587 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:15.365 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:12:51.588 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:15.367 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:12:51.590 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:15.563 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:12:51.821 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:15.577 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:12:51.799 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:15.584 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:12:51.807 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:48.247 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:13:24.511 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:48.251 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:13:24.515 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:48.298 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:13:24.562 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:48.304 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:13:24.526 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:48.317 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:13:24.539 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:48.320 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:13:24.543 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:48.334 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:13:24.557 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:48.367 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:13:24.589 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:48.368 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:13:24.591 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:25.704 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:18:01.969 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:25.723 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:18:01.983 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:25.724 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:18:01.946 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:25.727 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:18:01.950 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:25.728 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:18:01.951 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:25.737 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:18:01.960 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:25.921 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:18:02.141 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:25.930 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:18:02.153 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:25.987 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:18:02.251 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:10.364 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:21:46.627 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:10.367 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:21:46.632 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:10.384 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:21:46.607 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:10.393 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:21:46.616 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:10.401 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:21:46.624 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:10.426 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:21:46.649 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:10.503 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:21:46.726 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:10.541 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:21:46.764 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:10.645 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:21:46.909 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:35.893 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:23:12.157 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:35.899 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:23:12.162 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:35.914 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:23:12.137 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:35.941 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:23:12.163 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:35.961 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:23:12.183 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:35.967 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:23:12.190 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:36.020 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:23:12.284 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:36.040 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:23:12.262 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:36.046 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:23:12.267 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:50.069 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:28:26.332 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:50.119 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:28:26.342 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:50.121 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:28:26.344 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:50.139 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:28:26.403 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:50.202 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:28:26.424 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:50.225 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:28:26.448 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:50.389 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:28:26.653 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:50.408 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:28:26.631 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:50.502 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:28:26.724 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:42:32.416 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:30:08.680 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:42:32.432 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:30:08.696 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:42:32.471 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:30:08.693 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:42:32.506 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:30:08.729 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:42:32.515 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:30:08.732 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:42:32.528 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:30:08.742 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:42:32.541 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:30:08.804 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:42:32.583 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:30:08.805 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:42:32.587 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:30:08.810 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:54.562 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:38:30.826 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:54.594 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:38:30.816 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:54.599 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:38:30.822 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:54.624 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:38:30.888 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:54.644 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:38:30.867 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:54.647 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:38:30.870 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:54.849 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:38:31.072 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:54.853 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:38:31.075 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:54.893 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:38:31.158 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:51:10.943 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:38:47.207 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:51:10.952 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:38:47.216 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:51:10.959 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:38:47.182 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:51:10.961 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:38:47.184 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:51:10.965 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:38:47.188 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:51:10.965 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:38:47.188 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:51:11.000 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:38:47.223 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:51:11.000 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:38:47.223 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:51:11.034 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:38:47.298 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:24.926 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:44:01.190 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:24.950 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:44:01.204 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:24.957 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:44:01.180 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:24.958 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:44:01.181 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:24.967 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:44:01.190 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:24.969 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:44:01.192 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:25.173 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:44:01.438 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:25.208 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:44:01.431 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:25.282 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:44:01.504 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:59:17.328 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:46:53.592 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:59:17.335 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:46:53.600 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:59:17.355 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:46:53.578 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:59:17.388 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:46:53.611 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:59:17.390 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:46:53.613 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T12:59:17.405 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:46:53.628 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:59:17.482 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:46:53.746 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T12:59:17.532 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:46:53.754 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T12:59:17.565 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:46:53.788 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:35.161 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:49:11.425 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:35.171 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:49:11.435 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:35.173 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:49:11.395 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:35.177 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:49:11.400 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:35.187 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:49:11.409 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:35.190 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:49:11.413 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:35.291 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:49:11.514 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:35.322 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:49:11.544 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:35.325 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:49:11.590 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:47.409 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:54:23.673 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:47.426 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:54:23.691 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:47.433 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:54:23.656 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:47.435 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:54:23.658 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:47.439 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:54:23.662 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:47.443 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:54:23.665 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:47.661 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:54:23.884 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:47.693 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:54:23.915 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:47.707 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:54:23.971 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:44.454 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:55:20.718 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:44.514 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:55:20.779 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:44.534 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:55:20.756 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:44.536 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:55:20.759 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:44.542 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:55:20.765 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:44.545 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:55:20.768 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:44.598 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:55:20.862 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:44.629 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:55:20.851 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:44.634 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:55:20.857 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:59.772 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:59:36.036 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:59.791 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:59:36.046 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:59.792 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:59:36.014 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:59.798 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:59:36.021 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:59.811 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:59:36.034 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:59.814 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:59:36.037 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:59.992 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:03:59:36.215 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:12:00.030 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:03:59:36.294 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:12:00.030 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:03:59:36.253 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:06.654 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:03:42.918 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:06.680 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:03:42.928 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:06.689 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:03:42.911 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:06.729 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:03:42.946 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:06.729 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:03:42.948 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:06.748 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:03:42.971 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:06.858 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:03:43.122 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:06.863 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:03:43.086 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:06.913 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:03:43.136 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:17:11.121 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:04:47.385 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:17:11.131 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:04:47.396 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:17:11.141 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:04:47.363 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:17:11.144 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:04:47.367 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:17:11.156 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:04:47.369 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:17:11.156 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:04:47.372 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:17:11.211 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:04:47.476 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:17:11.222 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:04:47.445 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:17:11.227 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:04:47.450 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:23.311 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:09:59.575 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:23.338 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:09:59.550 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:23.338 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:09:59.552 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:23.338 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:09:59.556 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:23.338 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:09:59.560 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:23.338 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:09:59.588 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:23.614 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:09:59.836 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:23.622 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:09:59.887 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:23.626 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:09:59.849 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:24:28.973 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:12:05.238 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:24:28.990 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:12:05.213 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:24:29.005 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:12:05.270 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:24:29.055 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:12:05.277 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:24:29.060 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:12:05.283 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:24:29.071 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:12:05.294 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:24:29.114 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:12:05.379 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:24:29.152 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:12:05.375 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:24:29.171 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:12:05.394 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:32.955 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:15:09.219 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:32.974 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:15:09.195 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:32.974 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:15:09.238 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:32.979 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:15:09.202 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:32.980 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:15:09.203 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:32.988 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:15:09.206 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:33.113 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:15:09.335 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:33.143 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:15:09.366 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:33.193 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:15:09.458 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:46.042 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:20:22.306 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:46.059 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:20:22.272 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:46.059 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:20:22.279 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:46.059 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:20:22.278 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:46.061 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:20:22.326 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:46.067 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:20:22.290 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:46.274 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:20:22.496 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:46.329 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:20:22.552 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:46.331 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:20:22.596 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:51.083 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:20:27.348 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:51.147 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:20:27.366 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:51.147 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:20:27.368 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:51.150 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:20:27.414 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:51.173 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:20:27.395 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:51.180 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:20:27.402 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:51.187 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:20:27.410 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:51.191 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:20:27.456 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:51.211 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:20:27.434 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:52.493 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:25:28.757 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:52.521 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:25:28.761 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:52.569 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:25:28.789 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:52.575 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:25:28.797 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:52.586 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:25:28.809 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:52.592 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:25:28.811 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:52.725 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:25:28.984 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:52.753 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:25:28.976 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:52.840 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:25:29.063 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:57.384 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:25:33.649 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:57.398 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:25:33.663 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:57.407 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:25:33.629 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:57.409 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:25:33.632 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:57.414 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:25:33.636 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:57.414 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:25:33.637 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:57.443 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:25:33.666 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:57.446 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:25:33.669 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:57.492 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:25:33.757 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:41:13.687 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:28:49.950 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:41:13.710 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:28:49.963 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:41:13.762 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:28:49.984 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:41:13.779 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:28:50.002 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:41:13.782 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:28:50.005 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:41:13.782 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:28:50.005 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:41:13.805 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:28:50.070 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:41:13.889 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:28:50.111 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:41:13.912 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:28:50.134 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:43:09.284 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:30:45.546 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:43:09.429 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:30:45.536 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:43:09.429 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:30:45.543 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:43:09.430 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:30:45.542 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:43:09.430 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:30:45.549 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:43:09.431 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:30:45.559 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:43:09.447 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:30:45.669 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:43:09.452 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:30:45.714 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:43:09.461 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:30:45.682 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:21.535 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:35:57.800 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:21.544 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:35:57.808 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:21.561 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:35:57.783 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:21.567 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:35:57.789 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:21.574 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:35:57.796 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:21.583 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:35:57.805 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:21.781 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:35:58.045 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:21.795 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:35:58.018 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:21.824 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:35:58.047 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:35.678 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:37:11.942 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:35.686 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:37:11.950 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:35.745 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:37:11.968 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:35.747 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:37:11.970 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:35.747 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:37:11.970 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:35.762 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:37:11.977 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:35.784 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:37:12.048 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:35.796 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:37:12.018 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:35.831 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:37:12.053 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:32.657 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:41:08.922 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:32.667 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:41:08.932 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:32.677 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:41:08.894 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:32.677 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:41:08.897 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:32.677 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:41:08.897 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:32.683 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:41:08.905 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:32.857 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:41:09.073 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:32.917 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:41:09.182 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:32.957 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:41:09.180 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:54:31.909 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-12-09T13:54:31.917 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-12-09T13:54:31.918 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-12-09T13:54:31.918 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-12-09T13:54:31.918 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-12-09T13:54:31.918 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-12-09T13:54:31.918 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-12-09T13:54:32.309 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-12-09T13:54:32.589 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-12-09T13:54:33.119 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-12-09T13:54:33.148 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-12-09T13:54:33.167 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-12-09T13:54:33.167 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-12-09T13:54:33.167 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-12-09T13:54:33.168 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-12-09T13:54:33.168 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-12-09T13:54:33.168 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-12-09T13:54:33.168 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-12-09T13:54:33.171 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-12-09T13:54:33.171 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-12-09T13:54:33.172 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-12-09T13:54:33.172 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-12-09T13:54:33.173 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-12-09T13:54:33.173 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-12-09T13:54:33.173 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-12-09T13:54:33.185 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_child ... ok 2019-12-09T13:54:33.185 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-12-09T13:54:33.185 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-12-09T13:54:33.186 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-12-09T13:54:33.186 INFO:teuthology.orchestra.run.smithi035.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-12-09T13:54:33.196 INFO:teuthology.orchestra.run.smithi035.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-12-09T13:54:33.218 INFO:teuthology.orchestra.run.smithi035.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-12-09T13:54:33.218 INFO:teuthology.orchestra.run.smithi035.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-12-09T13:54:33.358 INFO:teuthology.orchestra.run.smithi035.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-12-09T13:54:33.358 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T13:54:33.359 INFO:teuthology.orchestra.run.smithi035.stderr:---------------------------------------------------------------------- 2019-12-09T13:54:33.359 INFO:teuthology.orchestra.run.smithi035.stderr:Ran 611 tests in 12768.894s 2019-12-09T13:54:33.359 INFO:teuthology.orchestra.run.smithi035.stderr: 2019-12-09T13:54:33.359 INFO:teuthology.orchestra.run.smithi035.stderr:OK (SKIP=75) 2019-12-09T13:54:33.404 DEBUG:tasks.s3tests:Scanning radosgw logs for leaked encryption keys... 2019-12-09T13:54:33.404 INFO:teuthology.orchestra.run.smithi035:> grep --binary-files=text pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= /var/log/ceph/rgw.ceph.client.0.log 2019-12-09T13:54:33.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T13:54:33.536 INFO:tasks.s3tests:Cleaning up boto... 2019-12-09T13:54:33.537 INFO:teuthology.orchestra.run.smithi035:> rm /home/ubuntu/cephtest/boto.cfg 2019-12-09T13:54:33.600 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid bar.client.0 --purge-data --cluster ceph 2019-12-09T13:54:33.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.740+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:33.767 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.765+0000 7f996b8afec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:33.767 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.765+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:33.778 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.776+0000 7f996b8afec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:33.778 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.776+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:33.783 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.781+0000 7f996b8afec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:33.783 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.781+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:33.791 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.789+0000 7f996b8afec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:33.958 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.956+0000 7f996b8afec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T13:54:33.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.956+0000 7f996b8afec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T13:54:33.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.956+0000 7f996b8afec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:54:33.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.956+0000 7f996b8afec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:54:33.959 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.956+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:33.969 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.967+0000 7f996b8afec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:33.970 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.967+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:33.982 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.980+0000 7f996b8afec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:54:33.982 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.980+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:33.990 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.988+0000 7f996b8afec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:33.990 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:33.988+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:34.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.009+0000 7f996b8afec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:54:34.011 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.009+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:34.017 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.015+0000 7f996b8afec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:34.017 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.015+0000 7f996b8afec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:54:34.017 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.015+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:34.022 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.020+0000 7f996b8afec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:34.022 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.020+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:34.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.031+0000 7f996b8afec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:34.033 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.031+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:34.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.044+0000 7f996b8afec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:54:34.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.044+0000 7f996b8afec0 20 zonegroup default 2019-12-09T13:54:34.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.044+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:34.052 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.049+0000 7f996b8afec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:34.052 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.049+0000 7f996b8afec0 10 Cannot find current period zone using local zone 2019-12-09T13:54:34.052 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.049+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:34.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.054+0000 7f996b8afec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:34.057 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.054+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:34.067 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.065+0000 7f996b8afec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:34.068 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.065+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:34.087 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.085+0000 7f996b8afec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:54:34.088 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.085+0000 7f996b8afec0 20 zone default found 2019-12-09T13:54:34.088 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.085+0000 7f996b8afec0 20 started sync module instance, tier type = 2019-12-09T13:54:34.088 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.085+0000 7f996b8afec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T13:54:34.692 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.690+0000 7f996b8afec0 20 add_watcher() i=0 2019-12-09T13:54:34.692 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.690+0000 7f996b8afec0 20 add_watcher() i=1 2019-12-09T13:54:34.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.690+0000 7f996b8afec0 20 add_watcher() i=2 2019-12-09T13:54:34.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.690+0000 7f996b8afec0 20 add_watcher() i=3 2019-12-09T13:54:34.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.690+0000 7f996b8afec0 20 add_watcher() i=4 2019-12-09T13:54:34.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.690+0000 7f996b8afec0 20 add_watcher() i=5 2019-12-09T13:54:34.693 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.690+0000 7f996b8afec0 20 add_watcher() i=6 2019-12-09T13:54:34.733 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.729+0000 7f996b8afec0 20 add_watcher() i=7 2019-12-09T13:54:34.733 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.729+0000 7f996b8afec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:54:34.733 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.730+0000 7f99297e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:54:34.754 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.752+0000 7f99592da700 0 -- 172.21.15.35:0/295959984 >> v2:172.21.15.22:3300/0 conn(0x565057e7c540 msgr2=0x565057e7c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T13:54:34.897 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:34.895+0000 7f9958ad9700 0 -- 172.21.15.35:0/295959984 >> [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] conn(0x565057e945c0 msgr2=0x565057e96a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:54:36.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.556+0000 7f989b7fe700 20 reqs_thread_entry: start 2019-12-09T13:54:36.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.556+0000 7f98997fa700 20 reqs_thread_entry: start 2019-12-09T13:54:36.559 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.557+0000 7f996b8afec0 20 init_complete bucket index max shards: 0 2019-12-09T13:54:36.560 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.557+0000 7f98877fe700 20 reqs_thread_entry: start 2019-12-09T13:54:36.560 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.557+0000 7f996b8afec0 20 get_system_obj_state: rctx=0x7fffe213eac8 obj=default.rgw.meta:users.uid:bar.client.0 state=0x565057efb090 s->prefetch_data=0 2019-12-09T13:54:36.560 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.557+0000 7f996b8afec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T13:54:36.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.579+0000 7f996b8afec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-12-09T13:54:36.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.579+0000 7f996b8afec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:54:36.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.579+0000 7f996b8afec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T13:54:36.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.579+0000 7f996b8afec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T13:54:36.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.579+0000 7f996b8afec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T13:54:36.583 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.579+0000 7f996b8afec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:36.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.606+0000 7f996b8afec0 20 rados_obj.operate() r=0 bl.length=446 2019-12-09T13:54:36.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.606+0000 7f996b8afec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-12-09T13:54:36.609 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.606+0000 7f996b8afec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:54:36.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.619+0000 7f996b8afec0 10 removing key index: EFFIUUJDNZHPZNOAEBSS 2019-12-09T13:54:36.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.619+0000 7f996b8afec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=170 2019-12-09T13:54:36.632 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.629+0000 7f994e7fc700 10 RGWWatcher::handle_notify() notify_id 588410519613 cookie 94903072070768 notifier 4694 bl.length()=170 2019-12-09T13:54:36.768 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.765+0000 7f996b8afec0 10 removing email index: bar.client.0+test@test.test 2019-12-09T13:54:36.768 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.765+0000 7f996b8afec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=178 2019-12-09T13:54:36.792 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.786+0000 7f994e7fc700 10 RGWWatcher::handle_notify() notify_id 588410519668 cookie 94903072063744 notifier 4694 bl.length()=178 2019-12-09T13:54:36.887 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.885+0000 7f996b8afec0 10 removing user buckets index 2019-12-09T13:54:36.887 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.885+0000 7f996b8afec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=169 2019-12-09T13:54:36.898 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:36.896+0000 7f994e7fc700 10 RGWWatcher::handle_notify() notify_id 588410520487 cookie 94903072065616 notifier 4694 bl.length()=169 2019-12-09T13:54:37.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.108+0000 7f996b8afec0 10 removing user index: bar.client.0 2019-12-09T13:54:37.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.108+0000 7f996b8afec0 10 removing default.rgw.meta+users.uid+bar.client.0 from cache 2019-12-09T13:54:37.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.108+0000 7f996b8afec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=161 2019-12-09T13:54:37.131 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.128+0000 7f994e7fc700 10 RGWWatcher::handle_notify() notify_id 588410519669 cookie 94903072063744 notifier 4694 bl.length()=161 2019-12-09T13:54:37.424 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.422+0000 7f996b8afec0 20 remove_watcher() i=0 2019-12-09T13:54:37.424 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.422+0000 7f996b8afec0 2 removed watcher, disabling cache 2019-12-09T13:54:37.463 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.460+0000 7f996b8afec0 20 remove_watcher() i=1 2019-12-09T13:54:37.502 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.499+0000 7f996b8afec0 20 remove_watcher() i=2 2019-12-09T13:54:37.538 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.536+0000 7f996b8afec0 20 remove_watcher() i=3 2019-12-09T13:54:37.606 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.604+0000 7f996b8afec0 20 remove_watcher() i=4 2019-12-09T13:54:37.667 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.665+0000 7f996b8afec0 20 remove_watcher() i=5 2019-12-09T13:54:37.743 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.741+0000 7f996b8afec0 20 remove_watcher() i=6 2019-12-09T13:54:37.788 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.785+0000 7f996b8afec0 20 remove_watcher() i=7 2019-12-09T13:54:37.796 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph 2019-12-09T13:54:37.948 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.946+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:37.988 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.985+0000 7f9e4ee65ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:37.988 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.985+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.997+0000 7f9e4ee65ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:38.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:37.997+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.007 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.005+0000 7f9e4ee65ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:38.008 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.005+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.016 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.014+0000 7f9e4ee65ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:38.247 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.245+0000 7f9e4ee65ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T13:54:38.247 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.245+0000 7f9e4ee65ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T13:54:38.248 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.245+0000 7f9e4ee65ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:54:38.248 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.245+0000 7f9e4ee65ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:54:38.248 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.245+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.265 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.263+0000 7f9e4ee65ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:38.265 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.263+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.282+0000 7f9e4ee65ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:54:38.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.282+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.295+0000 7f9e4ee65ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:38.297 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.295+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.313 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.310+0000 7f9e4ee65ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:54:38.313 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.310+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.318 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.316+0000 7f9e4ee65ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:38.319 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.316+0000 7f9e4ee65ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:54:38.319 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.316+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.324 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.322+0000 7f9e4ee65ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:38.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.322+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.334 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.331+0000 7f9e4ee65ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:38.334 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.331+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.354 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.352+0000 7f9e4ee65ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:54:38.354 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.352+0000 7f9e4ee65ec0 20 zonegroup default 2019-12-09T13:54:38.354 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.352+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.396 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.393+0000 7f9e4ee65ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:38.396 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.393+0000 7f9e4ee65ec0 10 Cannot find current period zone using local zone 2019-12-09T13:54:38.396 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.393+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.404 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.402+0000 7f9e4ee65ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:38.405 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.402+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.415 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.412+0000 7f9e4ee65ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:38.415 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.412+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:38.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.428+0000 7f9e4ee65ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:54:38.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.429+0000 7f9e4ee65ec0 20 zone default found 2019-12-09T13:54:38.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.429+0000 7f9e4ee65ec0 20 started sync module instance, tier type = 2019-12-09T13:54:38.431 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.429+0000 7f9e4ee65ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T13:54:38.862 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.860+0000 7f9e4ee65ec0 20 add_watcher() i=0 2019-12-09T13:54:38.862 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.860+0000 7f9e4ee65ec0 20 add_watcher() i=1 2019-12-09T13:54:38.862 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.860+0000 7f9e4ee65ec0 20 add_watcher() i=2 2019-12-09T13:54:38.862 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.860+0000 7f9e4ee65ec0 20 add_watcher() i=3 2019-12-09T13:54:38.863 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.860+0000 7f9e4ee65ec0 20 add_watcher() i=4 2019-12-09T13:54:38.863 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.860+0000 7f9e4ee65ec0 20 add_watcher() i=5 2019-12-09T13:54:38.872 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.869+0000 7f9e4ee65ec0 20 add_watcher() i=6 2019-12-09T13:54:38.900 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.897+0000 7f9e4ee65ec0 20 add_watcher() i=7 2019-12-09T13:54:38.900 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.897+0000 7f9e4ee65ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:54:38.900 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.898+0000 7f9e0cfe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:54:38.907 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.905+0000 7f9e3b88e700 0 -- 172.21.15.35:0/519976174 >> v1:172.21.15.35:6789/0 conn(0x558d188f0460 legacy=0x558d188f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:54:38.957 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:38.955+0000 7f9e3c890700 0 -- 172.21.15.35:0/429010131 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x558d188ef530 msgr2=0x558d188ef940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:54:40.580 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.577+0000 7f9d7effd700 20 reqs_thread_entry: start 2019-12-09T13:54:40.580 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.577+0000 7f9d7cff9700 20 reqs_thread_entry: start 2019-12-09T13:54:40.580 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.578+0000 7f9e4ee65ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:54:40.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.578+0000 7f9d72ffd700 20 reqs_thread_entry: start 2019-12-09T13:54:40.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.578+0000 7f9e4ee65ec0 20 get_system_obj_state: rctx=0x7ffe8a71a098 obj=default.rgw.meta:users.uid:foo.client.0 state=0x558d1896c800 s->prefetch_data=0 2019-12-09T13:54:40.582 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.578+0000 7f9e4ee65ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T13:54:40.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.607+0000 7f9e4ee65ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-12-09T13:54:40.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.607+0000 7f9e4ee65ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:54:40.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.607+0000 7f9e4ee65ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T13:54:40.610 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.607+0000 7f9e4ee65ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T13:54:40.611 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.607+0000 7f9e4ee65ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T13:54:40.611 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.607+0000 7f9e4ee65ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:40.639 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.636+0000 7f9e4ee65ec0 20 rados_obj.operate() r=0 bl.length=446 2019-12-09T13:54:40.639 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.637+0000 7f9e4ee65ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-12-09T13:54:40.639 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.637+0000 7f9e4ee65ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:54:40.669 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.667+0000 7f9e4ee65ec0 10 removing key index: AHJBSMMNUQWHBYXCCQJU 2019-12-09T13:54:40.670 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.667+0000 7f9e4ee65ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=170 2019-12-09T13:54:40.683 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:40.681+0000 7f9e31ffb700 10 RGWWatcher::handle_notify() notify_id 588410520488 cookie 94064490678704 notifier 4938 bl.length()=170 2019-12-09T13:54:41.346 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.343+0000 7f9e4ee65ec0 10 removing email index: foo.client.0+test@test.test 2019-12-09T13:54:41.346 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.343+0000 7f9e4ee65ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=178 2019-12-09T13:54:41.369 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.367+0000 7f9e31ffb700 10 RGWWatcher::handle_notify() notify_id 588410519628 cookie 94064490694368 notifier 4938 bl.length()=178 2019-12-09T13:54:41.565 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.563+0000 7f9e4ee65ec0 10 removing user buckets index 2019-12-09T13:54:41.565 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.563+0000 7f9e4ee65ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=169 2019-12-09T13:54:41.576 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.573+0000 7f9e31ffb700 10 RGWWatcher::handle_notify() notify_id 588410519670 cookie 94064490656240 notifier 4938 bl.length()=169 2019-12-09T13:54:41.847 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.844+0000 7f9e4ee65ec0 10 removing user index: foo.client.0 2019-12-09T13:54:41.847 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.844+0000 7f9e4ee65ec0 10 removing default.rgw.meta+users.uid+foo.client.0 from cache 2019-12-09T13:54:41.847 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.844+0000 7f9e4ee65ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=161 2019-12-09T13:54:41.864 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.862+0000 7f9e31ffb700 10 RGWWatcher::handle_notify() notify_id 588410519580 cookie 94064490652864 notifier 4938 bl.length()=161 2019-12-09T13:54:41.990 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.987+0000 7f9e4ee65ec0 20 remove_watcher() i=0 2019-12-09T13:54:41.990 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:41.987+0000 7f9e4ee65ec0 2 removed watcher, disabling cache 2019-12-09T13:54:42.020 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.018+0000 7f9e4ee65ec0 20 remove_watcher() i=1 2019-12-09T13:54:42.050 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.047+0000 7f9e4ee65ec0 20 remove_watcher() i=2 2019-12-09T13:54:42.118 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.116+0000 7f9e4ee65ec0 20 remove_watcher() i=3 2019-12-09T13:54:42.167 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.165+0000 7f9e4ee65ec0 20 remove_watcher() i=4 2019-12-09T13:54:42.224 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.221+0000 7f9e4ee65ec0 20 remove_watcher() i=5 2019-12-09T13:54:42.289 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.287+0000 7f9e4ee65ec0 20 remove_watcher() i=6 2019-12-09T13:54:42.340 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.338+0000 7f9e4ee65ec0 20 remove_watcher() i=7 2019-12-09T13:54:42.352 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid 'testx$tenanteduser.client.0' --purge-data --cluster ceph 2019-12-09T13:54:42.482 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.480+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.508 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.506+0000 7fbf9bb86ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:42.509 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.506+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.518 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.516+0000 7fbf9bb86ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:42.519 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.516+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.524 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.522+0000 7fbf9bb86ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:42.525 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.522+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.536 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.533+0000 7fbf9bb86ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:42.750 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.748+0000 7fbf9bb86ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T13:54:42.751 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.748+0000 7fbf9bb86ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T13:54:42.751 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.748+0000 7fbf9bb86ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:54:42.751 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.748+0000 7fbf9bb86ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:54:42.751 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.748+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.763 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.761+0000 7fbf9bb86ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:42.763 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.761+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.794 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.791+0000 7fbf9bb86ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:54:42.795 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.791+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.821 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.817+0000 7fbf9bb86ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:42.821 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.817+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.842 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.840+0000 7fbf9bb86ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:54:42.842 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.840+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.849 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.847+0000 7fbf9bb86ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:42.849 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.847+0000 7fbf9bb86ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:54:42.849 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.847+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.854+0000 7fbf9bb86ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:42.857 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.854+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.865 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.863+0000 7fbf9bb86ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:42.865 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.863+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.881 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.879+0000 7fbf9bb86ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:54:42.882 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.879+0000 7fbf9bb86ec0 20 zonegroup default 2019-12-09T13:54:42.882 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.879+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.886+0000 7fbf9bb86ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:42.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.886+0000 7fbf9bb86ec0 10 Cannot find current period zone using local zone 2019-12-09T13:54:42.888 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.886+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.893 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.891+0000 7fbf9bb86ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:42.893 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.891+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.902 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.900+0000 7fbf9bb86ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:42.902 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.900+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:42.917 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.915+0000 7fbf9bb86ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:54:42.918 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.915+0000 7fbf9bb86ec0 20 zone default found 2019-12-09T13:54:42.918 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.915+0000 7fbf9bb86ec0 20 started sync module instance, tier type = 2019-12-09T13:54:42.919 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:42.915+0000 7fbf9bb86ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T13:54:43.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.282+0000 7fbf9bb86ec0 20 add_watcher() i=0 2019-12-09T13:54:43.284 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.282+0000 7fbf9bb86ec0 20 add_watcher() i=1 2019-12-09T13:54:43.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.282+0000 7fbf9bb86ec0 20 add_watcher() i=2 2019-12-09T13:54:43.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.282+0000 7fbf9bb86ec0 20 add_watcher() i=3 2019-12-09T13:54:43.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.282+0000 7fbf9bb86ec0 20 add_watcher() i=4 2019-12-09T13:54:43.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.282+0000 7fbf9bb86ec0 20 add_watcher() i=5 2019-12-09T13:54:43.285 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.282+0000 7fbf9bb86ec0 20 add_watcher() i=6 2019-12-09T13:54:43.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.313+0000 7fbf9bb86ec0 20 add_watcher() i=7 2019-12-09T13:54:43.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.313+0000 7fbf9bb86ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:54:43.316 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.313+0000 7fbf58fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:54:43.325 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.323+0000 7fbf88db0700 0 -- 172.21.15.35:0/787275478 >> v1:172.21.15.22:6789/0 conn(0x56088bd2d500 legacy=0x56088bd2f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:54:43.383 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:43.381+0000 7fbf88db0700 0 -- 172.21.15.35:0/1881966551 >> [v2:172.21.15.35:6808/9789,v1:172.21.15.35:6809/9789] conn(0x56088bd425c0 msgr2=0x56088bd449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:54:45.313 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.310+0000 7fbecaffd700 20 reqs_thread_entry: start 2019-12-09T13:54:45.314 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.310+0000 7fbec8ff9700 20 reqs_thread_entry: start 2019-12-09T13:54:45.314 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.311+0000 7fbf9bb86ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:54:45.314 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.311+0000 7fbebeffd700 20 reqs_thread_entry: start 2019-12-09T13:54:45.315 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.311+0000 7fbf9bb86ec0 20 get_system_obj_state: rctx=0x7ffde0e3ba48 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x56088bdaf060 s->prefetch_data=0 2019-12-09T13:54:45.315 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.311+0000 7fbf9bb86ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T13:54:45.328 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.325+0000 7fbf9bb86ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-12-09T13:54:45.328 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.325+0000 7fbf9bb86ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T13:54:45.329 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.325+0000 7fbf9bb86ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T13:54:45.329 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.325+0000 7fbf9bb86ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T13:54:45.329 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.325+0000 7fbf9bb86ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T13:54:45.329 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.325+0000 7fbf9bb86ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:45.341 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.339+0000 7fbf9bb86ec0 20 rados_obj.operate() r=0 bl.length=505 2019-12-09T13:54:45.341 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.339+0000 7fbf9bb86ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-12-09T13:54:45.342 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.339+0000 7fbf9bb86ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T13:54:45.359 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.349+0000 7fbf9bb86ec0 10 removing key index: YTABYFXHGYOWVVUVVLWN 2019-12-09T13:54:45.359 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.349+0000 7fbf9bb86ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=170 2019-12-09T13:54:45.381 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.371+0000 7fbf71ffb700 10 RGWWatcher::handle_notify() notify_id 588410519688 cookie 94594705479808 notifier 4709 bl.length()=170 2019-12-09T13:54:45.672 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.668+0000 7fbf9bb86ec0 10 removing email index: testx$tenanteduser.client.0+test@test.test 2019-12-09T13:54:45.672 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.668+0000 7fbf9bb86ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=193 2019-12-09T13:54:45.696 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.691+0000 7fbf71ffb700 10 RGWWatcher::handle_notify() notify_id 588410519629 cookie 94594705513776 notifier 4709 bl.length()=193 2019-12-09T13:54:45.828 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.824+0000 7fbf9bb86ec0 10 removing user buckets index 2019-12-09T13:54:45.828 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.824+0000 7fbf9bb86ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=184 2019-12-09T13:54:45.856 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.852+0000 7fbf71ffb700 10 RGWWatcher::handle_notify() notify_id 588410519630 cookie 94594705512976 notifier 4709 bl.length()=184 2019-12-09T13:54:46.001 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.998+0000 7fbf9bb86ec0 10 removing user index: testx$tenanteduser.client.0 2019-12-09T13:54:46.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.998+0000 7fbf9bb86ec0 10 removing default.rgw.meta+users.uid+testx$tenanteduser.client.0 from cache 2019-12-09T13:54:46.002 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:45.998+0000 7fbf9bb86ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=176 2019-12-09T13:54:46.010 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:46.006+0000 7fbf71ffb700 10 RGWWatcher::handle_notify() notify_id 588410520489 cookie 94594705497584 notifier 4709 bl.length()=176 2019-12-09T13:54:46.660 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:46.656+0000 7fbf9bb86ec0 20 remove_watcher() i=0 2019-12-09T13:54:46.660 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:46.657+0000 7fbf9bb86ec0 2 removed watcher, disabling cache 2019-12-09T13:54:46.735 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:46.731+0000 7fbf9bb86ec0 20 remove_watcher() i=1 2019-12-09T13:54:46.806 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:46.803+0000 7fbf9bb86ec0 20 remove_watcher() i=2 2019-12-09T13:54:46.868 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:46.865+0000 7fbf9bb86ec0 20 remove_watcher() i=3 2019-12-09T13:54:46.910 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:46.907+0000 7fbf9bb86ec0 20 remove_watcher() i=4 2019-12-09T13:54:46.946 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:46.943+0000 7fbf9bb86ec0 20 remove_watcher() i=5 2019-12-09T13:54:47.004 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:47.001+0000 7fbf9bb86ec0 20 remove_watcher() i=6 2019-12-09T13:54:47.060 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:47.057+0000 7fbf9bb86ec0 20 remove_watcher() i=7 2019-12-09T13:54:47.073 INFO:tasks.s3tests:Removing s3-tests... 2019-12-09T13:54:47.073 INFO:teuthology.orchestra.run.smithi035:> rm -rf /home/ubuntu/cephtest/s3-tests 2019-12-09T13:54:47.229 INFO:teuthology.run_tasks:Running task swift... 2019-12-09T13:54:47.250 INFO:tasks.swift:clients=['client.0'] 2019-12-09T13:54:47.250 INFO:tasks.swift:Downloading swift... 2019-12-09T13:54:47.251 INFO:teuthology.orchestra.run.smithi035:> git clone -b ceph-master git://git.ceph.com/git/swift.git /home/ubuntu/cephtest/swift 2019-12-09T13:54:47.368 INFO:teuthology.orchestra.run.smithi035.stdout:Cloning into '/home/ubuntu/cephtest/swift'... 2019-12-09T13:54:47.836 INFO:tasks.swift:Creating rgw users... 2019-12-09T13:54:47.948 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser foo.client.0:foo --display-name 'Mr. foo.client.0 foo' --secret PD8RoxpyAcG31VYLdpNsGb++Vl+ccTSHVge6oJmCP2MmxUfCAk0jZA== --email foo.client.0+test@test.test --key-type swift --access full 2019-12-09T13:54:48.099 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.096+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.136 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.132+0000 7f1822c70ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:48.136 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.132+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.148 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.145+0000 7f1822c70ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:48.148 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.145+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.154 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.151+0000 7f1822c70ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:48.154 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.151+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.166 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.163+0000 7f1822c70ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:48.403 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.400+0000 7f1822c70ec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T13:54:48.403 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.400+0000 7f1822c70ec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T13:54:48.404 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.400+0000 7f1822c70ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:54:48.404 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.400+0000 7f1822c70ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:54:48.404 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.400+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.417 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.413+0000 7f1822c70ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:48.417 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.413+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.432 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.429+0000 7f1822c70ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:54:48.432 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.429+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.444 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.441+0000 7f1822c70ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:48.445 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.441+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.463 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.459+0000 7f1822c70ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:54:48.463 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.459+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.469 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.466+0000 7f1822c70ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:48.470 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.466+0000 7f1822c70ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:54:48.470 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.466+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.476 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.473+0000 7f1822c70ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:48.477 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.473+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.486 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.483+0000 7f1822c70ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:48.487 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.483+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.502 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.499+0000 7f1822c70ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:54:48.502 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.499+0000 7f1822c70ec0 20 zonegroup default 2019-12-09T13:54:48.503 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.499+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.516 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.508+0000 7f1822c70ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:48.516 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.508+0000 7f1822c70ec0 10 Cannot find current period zone using local zone 2019-12-09T13:54:48.516 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.508+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.517 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.514+0000 7f1822c70ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:48.517 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.514+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.538 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.529+0000 7f1822c70ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:48.538 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.529+0000 7f1822c70ec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:48.557 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.553+0000 7f1822c70ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:54:48.557 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.553+0000 7f1822c70ec0 20 zone default found 2019-12-09T13:54:48.557 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.553+0000 7f1822c70ec0 20 started sync module instance, tier type = 2019-12-09T13:54:48.557 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:48.553+0000 7f1822c70ec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T13:54:49.045 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.042+0000 7f1822c70ec0 20 add_watcher() i=0 2019-12-09T13:54:49.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.042+0000 7f1822c70ec0 20 add_watcher() i=1 2019-12-09T13:54:49.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.042+0000 7f1822c70ec0 20 add_watcher() i=2 2019-12-09T13:54:49.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.042+0000 7f1822c70ec0 20 add_watcher() i=3 2019-12-09T13:54:49.046 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.042+0000 7f1822c70ec0 20 add_watcher() i=4 2019-12-09T13:54:49.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.042+0000 7f1822c70ec0 20 add_watcher() i=5 2019-12-09T13:54:49.047 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.042+0000 7f1822c70ec0 20 add_watcher() i=6 2019-12-09T13:54:49.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.088+0000 7f1822c70ec0 20 add_watcher() i=7 2019-12-09T13:54:49.091 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.088+0000 7f1822c70ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:54:49.092 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.088+0000 7f1763fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:54:49.095 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.091+0000 7f181069b700 0 -- 172.21.15.35:0/2878516278 >> v2:172.21.15.35:3300/0 conn(0x561bd4636500 msgr2=0x561bd4636910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T13:54:49.139 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:49.135+0000 7f181069b700 0 -- 172.21.15.35:0/221971054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561bd463e6f0 msgr2=0x561bd463eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:54:50.672 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.669+0000 7f174e7fc700 20 reqs_thread_entry: start 2019-12-09T13:54:50.672 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.669+0000 7f1747fff700 20 reqs_thread_entry: start 2019-12-09T13:54:50.672 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.670+0000 7f1822c70ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:54:50.673 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.670+0000 7f17467fc700 20 reqs_thread_entry: start 2019-12-09T13:54:50.673 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.670+0000 7f1822c70ec0 20 get_system_obj_state: rctx=0x7fffc74153f8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x561bd46ba810 s->prefetch_data=0 2019-12-09T13:54:50.673 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.670+0000 7f1822c70ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T13:54:50.688 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.677+0000 7f1822c70ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-12-09T13:54:50.689 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.677+0000 7f1822c70ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:54:50.689 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.677+0000 7f1822c70ec0 20 get_system_obj_state: rctx=0x7fffc7415408 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x561bd46bc5a0 s->prefetch_data=0 2019-12-09T13:54:50.689 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.677+0000 7f1822c70ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-12-09T13:54:50.689 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.682+0000 7f1822c70ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-12-09T13:54:50.689 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.682+0000 7f1822c70ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T13:54:50.689 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.682+0000 7f1822c70ec0 20 get_system_obj_state: rctx=0x7fffc7415018 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x561bd46bc5a0 s->prefetch_data=0 2019-12-09T13:54:50.690 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.682+0000 7f1822c70ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : miss 2019-12-09T13:54:50.690 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.687+0000 7f1822c70ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-12-09T13:54:50.690 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.687+0000 7f1822c70ec0 10 adding default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T13:54:50.690 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.687+0000 7f1822c70ec0 20 get_system_obj_state: rctx=0x7fffc7415198 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x561bd46bc850 s->prefetch_data=0 2019-12-09T13:54:50.691 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.687+0000 7f1822c70ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : type miss (requested=0x6, cached=0x0) 2019-12-09T13:54:50.694 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.692+0000 7f1822c70ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-12-09T13:54:50.694 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.692+0000 7f1822c70ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T13:54:50.766 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.764+0000 7f1822c70ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:54:50.766 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.764+0000 7f1822c70ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:54:50.767 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.764+0000 7f1822c70ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=612 2019-12-09T13:54:50.794 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.792+0000 7f18057fa700 10 RGWWatcher::handle_notify() notify_id 588410519581 cookie 94677527295328 notifier 4962 bl.length()=612 2019-12-09T13:54:50.795 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.792+0000 7f18057fa700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:54:50.795 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:50.792+0000 7f18057fa700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:54:51.126 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.124+0000 7f1822c70ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T13:54:51.126 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.124+0000 7f1822c70ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T13:54:51.127 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.124+0000 7f1822c70ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-12-09T13:54:51.137 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.135+0000 7f18057fa700 10 RGWWatcher::handle_notify() notify_id 588410519631 cookie 94677527323696 notifier 4962 bl.length()=194 2019-12-09T13:54:51.137 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.135+0000 7f18057fa700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T13:54:51.138 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.135+0000 7f18057fa700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T13:54:51.280 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.278+0000 7f1822c70ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-12-09T13:54:51.280 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.278+0000 7f1822c70ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T13:54:51.280 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.278+0000 7f1822c70ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=183 2019-12-09T13:54:51.293 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.290+0000 7f18057fa700 10 RGWWatcher::handle_notify() notify_id 588410519671 cookie 94677527297376 notifier 4962 bl.length()=183 2019-12-09T13:54:51.293 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.290+0000 7f18057fa700 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-12-09T13:54:51.293 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.290+0000 7f18057fa700 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T13:54:51.611 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.609+0000 7f1822c70ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:54:51.611 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.609+0000 7f1822c70ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:54:51.611 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.609+0000 7f1822c70ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=636 2019-12-09T13:54:51.624 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.622+0000 7f18057fa700 10 RGWWatcher::handle_notify() notify_id 588410519582 cookie 94677527295328 notifier 4962 bl.length()=636 2019-12-09T13:54:51.624 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.622+0000 7f18057fa700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:54:51.624 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.622+0000 7f18057fa700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:54:51.793 INFO:teuthology.orchestra.run.smithi035.stdout:{ 2019-12-09T13:54:51.793 INFO:teuthology.orchestra.run.smithi035.stdout: "user_id": "foo.client.0", 2019-12-09T13:54:51.794 INFO:teuthology.orchestra.run.smithi035.stdout: "display_name": "Mr. foo.client.0 foo", 2019-12-09T13:54:51.794 INFO:teuthology.orchestra.run.smithi035.stdout: "email": "foo.client.0+test@test.test", 2019-12-09T13:54:51.794 INFO:teuthology.orchestra.run.smithi035.stdout: "suspended": 0, 2019-12-09T13:54:51.794 INFO:teuthology.orchestra.run.smithi035.stdout: "max_buckets": 1000, 2019-12-09T13:54:51.794 INFO:teuthology.orchestra.run.smithi035.stdout: "subusers": [ 2019-12-09T13:54:51.794 INFO:teuthology.orchestra.run.smithi035.stdout: { 2019-12-09T13:54:51.794 INFO:teuthology.orchestra.run.smithi035.stdout: "id": "foo.client.0:foo", 2019-12-09T13:54:51.795 INFO:teuthology.orchestra.run.smithi035.stdout: "permissions": "full-control" 2019-12-09T13:54:51.795 INFO:teuthology.orchestra.run.smithi035.stdout: } 2019-12-09T13:54:51.795 INFO:teuthology.orchestra.run.smithi035.stdout: ], 2019-12-09T13:54:51.795 INFO:teuthology.orchestra.run.smithi035.stdout: "keys": [], 2019-12-09T13:54:51.795 INFO:teuthology.orchestra.run.smithi035.stdout: "swift_keys": [ 2019-12-09T13:54:51.795 INFO:teuthology.orchestra.run.smithi035.stdout: { 2019-12-09T13:54:51.795 INFO:teuthology.orchestra.run.smithi035.stdout: "user": "foo.client.0:foo", 2019-12-09T13:54:51.796 INFO:teuthology.orchestra.run.smithi035.stdout: "secret_key": "PD8RoxpyAcG31VYLdpNsGb++Vl+ccTSHVge6oJmCP2MmxUfCAk0jZA==" 2019-12-09T13:54:51.796 INFO:teuthology.orchestra.run.smithi035.stdout: } 2019-12-09T13:54:51.796 INFO:teuthology.orchestra.run.smithi035.stdout: ], 2019-12-09T13:54:51.796 INFO:teuthology.orchestra.run.smithi035.stdout: "caps": [], 2019-12-09T13:54:51.796 INFO:teuthology.orchestra.run.smithi035.stdout: "op_mask": "read, write, delete", 2019-12-09T13:54:51.796 INFO:teuthology.orchestra.run.smithi035.stdout: "default_placement": "", 2019-12-09T13:54:51.796 INFO:teuthology.orchestra.run.smithi035.stdout: "default_storage_class": "", 2019-12-09T13:54:51.797 INFO:teuthology.orchestra.run.smithi035.stdout: "placement_tags": [], 2019-12-09T13:54:51.797 INFO:teuthology.orchestra.run.smithi035.stdout: "bucket_quota": { 2019-12-09T13:54:51.797 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T13:54:51.797 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T13:54:51.797 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T13:54:51.797 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T13:54:51.797 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T13:54:51.798 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T13:54:51.798 INFO:teuthology.orchestra.run.smithi035.stdout: "user_quota": { 2019-12-09T13:54:51.798 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T13:54:51.798 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T13:54:51.798 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T13:54:51.798 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T13:54:51.798 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T13:54:51.799 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T13:54:51.799 INFO:teuthology.orchestra.run.smithi035.stdout: "temp_url_keys": [], 2019-12-09T13:54:51.799 INFO:teuthology.orchestra.run.smithi035.stdout: "type": "rgw", 2019-12-09T13:54:51.799 INFO:teuthology.orchestra.run.smithi035.stdout: "mfa_ids": [] 2019-12-09T13:54:51.799 INFO:teuthology.orchestra.run.smithi035.stdout:} 2019-12-09T13:54:51.799 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T13:54:51.836 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.834+0000 7f1822c70ec0 20 remove_watcher() i=0 2019-12-09T13:54:51.836 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.834+0000 7f1822c70ec0 2 removed watcher, disabling cache 2019-12-09T13:54:51.914 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.904+0000 7f1822c70ec0 20 remove_watcher() i=1 2019-12-09T13:54:51.949 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.947+0000 7f1822c70ec0 20 remove_watcher() i=2 2019-12-09T13:54:52.000 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:51.997+0000 7f1822c70ec0 20 remove_watcher() i=3 2019-12-09T13:54:52.034 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.031+0000 7f1822c70ec0 20 remove_watcher() i=4 2019-12-09T13:54:52.070 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.067+0000 7f1822c70ec0 20 remove_watcher() i=5 2019-12-09T13:54:52.111 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.109+0000 7f1822c70ec0 20 remove_watcher() i=6 2019-12-09T13:54:52.160 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.157+0000 7f1822c70ec0 20 remove_watcher() i=7 2019-12-09T13:54:52.169 INFO:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser bar.client.0:bar --display-name 'Mr. bar.client.0 bar' --secret 8yDJcz0BN03/84IMXlMoYAegPJX0gIvQwVjMfFxv9T6aojtinwc4xQ== --email bar.client.0+test@test.test --key-type swift --access full 2019-12-09T13:54:52.300 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.298+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.323 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.321+0000 7f491b75fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:52.323 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.321+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.333 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.331+0000 7f491b75fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:52.333 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.331+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.339 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.337+0000 7f491b75fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:52.339 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.337+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.350 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.348+0000 7f491b75fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:52.526 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.524+0000 7f491b75fec0 20 RGWRados::pool_iterate: got zone_info.d0d8af73-443a-4cb3-a612-7fc7fb9ae85c 2019-12-09T13:54:52.526 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.524+0000 7f491b75fec0 20 RGWRados::pool_iterate: got zonegroup_info.4984ff48-151f-4571-95c2-e1c44ebf7ca3 2019-12-09T13:54:52.527 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.524+0000 7f491b75fec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:54:52.527 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.524+0000 7f491b75fec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:54:52.527 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.524+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.537 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.535+0000 7f491b75fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:52.537 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.535+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.557 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.551+0000 7f491b75fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:54:52.557 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.551+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.562 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.560+0000 7f491b75fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:52.562 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.560+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.574 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.572+0000 7f491b75fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:54:52.575 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.572+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.580 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.577+0000 7f491b75fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:52.580 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.577+0000 7f491b75fec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:54:52.580 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.577+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.586 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.583+0000 7f491b75fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:52.586 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.583+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.594 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.592+0000 7f491b75fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:52.595 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.592+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.612 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.610+0000 7f491b75fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:54:52.612 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.610+0000 7f491b75fec0 20 zonegroup default 2019-12-09T13:54:52.612 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.610+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.621 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.618+0000 7f491b75fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.618+0000 7f491b75fec0 10 Cannot find current period zone using local zone 2019-12-09T13:54:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.618+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.625 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.623+0000 7f491b75fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:54:52.626 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.623+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.637 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.634+0000 7f491b75fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:54:52.637 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.635+0000 7f491b75fec0 20 rados->read ofs=0 len=0 2019-12-09T13:54:52.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.649+0000 7f491b75fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:54:52.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.649+0000 7f491b75fec0 20 zone default found 2019-12-09T13:54:52.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.649+0000 7f491b75fec0 20 started sync module instance, tier type = 2019-12-09T13:54:52.652 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:52.649+0000 7f491b75fec0 20 started zone id=d0d8af73-443a-4cb3-a612-7fc7fb9ae85c (name=default) with tier type = 2019-12-09T13:54:53.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.171+0000 7f491b75fec0 20 add_watcher() i=0 2019-12-09T13:54:53.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.171+0000 7f491b75fec0 20 add_watcher() i=1 2019-12-09T13:54:53.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.171+0000 7f491b75fec0 20 add_watcher() i=2 2019-12-09T13:54:53.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.171+0000 7f491b75fec0 20 add_watcher() i=3 2019-12-09T13:54:53.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.171+0000 7f491b75fec0 20 add_watcher() i=4 2019-12-09T13:54:53.174 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.171+0000 7f491b75fec0 20 add_watcher() i=5 2019-12-09T13:54:53.213 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.203+0000 7f491b75fec0 20 add_watcher() i=6 2019-12-09T13:54:53.214 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.212+0000 7f491b75fec0 20 add_watcher() i=7 2019-12-09T13:54:53.214 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.212+0000 7f491b75fec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:54:53.214 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.212+0000 7f48d8fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:54:53.221 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.219+0000 7f4908989700 0 -- 172.21.15.35:0/283332549 >> v2:172.21.15.35:3300/0 conn(0x55b3e129ab90 msgr2=0x55b3e129cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T13:54:53.424 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.422+0000 7f4908989700 0 --2- 172.21.15.35:0/283332549 >> v2:172.21.15.35:3300/0 conn(0x55b3e129ab90 0x55b3e129cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T13:54:53.517 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:53.515+0000 7f4908989700 0 -- 172.21.15.35:0/1521855260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b3e12a0de0 msgr2=0x55b3e12a3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:54:55.602 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.599+0000 7f484affd700 20 reqs_thread_entry: start 2019-12-09T13:54:55.602 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.600+0000 7f4848ff9700 20 reqs_thread_entry: start 2019-12-09T13:54:55.602 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.600+0000 7f491b75fec0 20 init_complete bucket index max shards: 0 2019-12-09T13:54:55.603 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.600+0000 7f483effd700 20 reqs_thread_entry: start 2019-12-09T13:54:55.603 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.600+0000 7f491b75fec0 20 get_system_obj_state: rctx=0x7ffc4d1643f8 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55b3e1314ad0 s->prefetch_data=0 2019-12-09T13:54:55.603 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.600+0000 7f491b75fec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T13:54:55.617 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.614+0000 7f491b75fec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-12-09T13:54:55.617 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.614+0000 7f491b75fec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:54:55.617 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.614+0000 7f491b75fec0 20 get_system_obj_state: rctx=0x7ffc4d164408 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x55b3e1316860 s->prefetch_data=0 2019-12-09T13:54:55.617 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.614+0000 7f491b75fec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-12-09T13:54:55.624 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.622+0000 7f491b75fec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-12-09T13:54:55.624 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.622+0000 7f491b75fec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T13:54:55.624 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.622+0000 7f491b75fec0 20 get_system_obj_state: rctx=0x7ffc4d164018 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x55b3e1316860 s->prefetch_data=0 2019-12-09T13:54:55.624 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.622+0000 7f491b75fec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : miss 2019-12-09T13:54:55.642 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.632+0000 7f491b75fec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-12-09T13:54:55.643 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.632+0000 7f491b75fec0 10 adding default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T13:54:55.643 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.632+0000 7f491b75fec0 20 get_system_obj_state: rctx=0x7ffc4d164198 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x55b3e1316b10 s->prefetch_data=0 2019-12-09T13:54:55.643 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.632+0000 7f491b75fec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : type miss (requested=0x6, cached=0x0) 2019-12-09T13:54:55.643 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.638+0000 7f491b75fec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-12-09T13:54:55.643 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.638+0000 7f491b75fec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T13:54:55.702 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.700+0000 7f491b75fec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:54:55.702 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.700+0000 7f491b75fec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:54:55.702 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.700+0000 7f491b75fec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=612 2019-12-09T13:54:55.715 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.713+0000 7f48f1ffb700 10 RGWWatcher::handle_notify() notify_id 588410519672 cookie 94231064999024 notifier 4727 bl.length()=612 2019-12-09T13:54:55.715 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.713+0000 7f48f1ffb700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:54:55.716 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:55.713+0000 7f48f1ffb700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:54:56.013 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.011+0000 7f491b75fec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T13:54:56.013 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.011+0000 7f491b75fec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T13:54:56.013 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.011+0000 7f491b75fec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-12-09T13:54:56.023 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.021+0000 7f48f1ffb700 10 RGWWatcher::handle_notify() notify_id 588410519673 cookie 94231064999024 notifier 4727 bl.length()=194 2019-12-09T13:54:56.023 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.021+0000 7f48f1ffb700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T13:54:56.023 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.021+0000 7f48f1ffb700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T13:54:56.235 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.232+0000 7f491b75fec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-12-09T13:54:56.235 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.232+0000 7f491b75fec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T13:54:56.235 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.232+0000 7f491b75fec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=183 2019-12-09T13:54:56.254 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.252+0000 7f48f1ffb700 10 RGWWatcher::handle_notify() notify_id 588410520490 cookie 94231065021424 notifier 4727 bl.length()=183 2019-12-09T13:54:56.254 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.252+0000 7f48f1ffb700 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-12-09T13:54:56.254 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.252+0000 7f48f1ffb700 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T13:54:56.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.431+0000 7f491b75fec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:54:56.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.431+0000 7f491b75fec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:54:56.433 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.431+0000 7f491b75fec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=636 2019-12-09T13:54:56.447 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.445+0000 7f48f1ffb700 10 RGWWatcher::handle_notify() notify_id 588410519674 cookie 94231064999024 notifier 4727 bl.length()=636 2019-12-09T13:54:56.448 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.445+0000 7f48f1ffb700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:54:56.448 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.445+0000 7f48f1ffb700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:54:56.625 INFO:teuthology.orchestra.run.smithi035.stdout:{ 2019-12-09T13:54:56.625 INFO:teuthology.orchestra.run.smithi035.stdout: "user_id": "bar.client.0", 2019-12-09T13:54:56.625 INFO:teuthology.orchestra.run.smithi035.stdout: "display_name": "Mr. bar.client.0 bar", 2019-12-09T13:54:56.625 INFO:teuthology.orchestra.run.smithi035.stdout: "email": "bar.client.0+test@test.test", 2019-12-09T13:54:56.625 INFO:teuthology.orchestra.run.smithi035.stdout: "suspended": 0, 2019-12-09T13:54:56.626 INFO:teuthology.orchestra.run.smithi035.stdout: "max_buckets": 1000, 2019-12-09T13:54:56.626 INFO:teuthology.orchestra.run.smithi035.stdout: "subusers": [ 2019-12-09T13:54:56.626 INFO:teuthology.orchestra.run.smithi035.stdout: { 2019-12-09T13:54:56.626 INFO:teuthology.orchestra.run.smithi035.stdout: "id": "bar.client.0:bar", 2019-12-09T13:54:56.626 INFO:teuthology.orchestra.run.smithi035.stdout: "permissions": "full-control" 2019-12-09T13:54:56.627 INFO:teuthology.orchestra.run.smithi035.stdout: } 2019-12-09T13:54:56.627 INFO:teuthology.orchestra.run.smithi035.stdout: ], 2019-12-09T13:54:56.627 INFO:teuthology.orchestra.run.smithi035.stdout: "keys": [], 2019-12-09T13:54:56.627 INFO:teuthology.orchestra.run.smithi035.stdout: "swift_keys": [ 2019-12-09T13:54:56.627 INFO:teuthology.orchestra.run.smithi035.stdout: { 2019-12-09T13:54:56.627 INFO:teuthology.orchestra.run.smithi035.stdout: "user": "bar.client.0:bar", 2019-12-09T13:54:56.627 INFO:teuthology.orchestra.run.smithi035.stdout: "secret_key": "8yDJcz0BN03/84IMXlMoYAegPJX0gIvQwVjMfFxv9T6aojtinwc4xQ==" 2019-12-09T13:54:56.628 INFO:teuthology.orchestra.run.smithi035.stdout: } 2019-12-09T13:54:56.628 INFO:teuthology.orchestra.run.smithi035.stdout: ], 2019-12-09T13:54:56.629 INFO:teuthology.orchestra.run.smithi035.stdout: "caps": [], 2019-12-09T13:54:56.629 INFO:teuthology.orchestra.run.smithi035.stdout: "op_mask": "read, write, delete", 2019-12-09T13:54:56.629 INFO:teuthology.orchestra.run.smithi035.stdout: "default_placement": "", 2019-12-09T13:54:56.630 INFO:teuthology.orchestra.run.smithi035.stdout: "default_storage_class": "", 2019-12-09T13:54:56.630 INFO:teuthology.orchestra.run.smithi035.stdout: "placement_tags": [], 2019-12-09T13:54:56.630 INFO:teuthology.orchestra.run.smithi035.stdout: "bucket_quota": { 2019-12-09T13:54:56.630 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T13:54:56.630 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T13:54:56.630 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T13:54:56.630 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T13:54:56.630 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T13:54:56.631 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T13:54:56.636 INFO:teuthology.orchestra.run.smithi035.stdout: "user_quota": { 2019-12-09T13:54:56.636 INFO:teuthology.orchestra.run.smithi035.stdout: "enabled": false, 2019-12-09T13:54:56.637 INFO:teuthology.orchestra.run.smithi035.stdout: "check_on_raw": false, 2019-12-09T13:54:56.637 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size": -1, 2019-12-09T13:54:56.637 INFO:teuthology.orchestra.run.smithi035.stdout: "max_size_kb": 0, 2019-12-09T13:54:56.637 INFO:teuthology.orchestra.run.smithi035.stdout: "max_objects": -1 2019-12-09T13:54:56.637 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2019-12-09T13:54:56.637 INFO:teuthology.orchestra.run.smithi035.stdout: "temp_url_keys": [], 2019-12-09T13:54:56.637 INFO:teuthology.orchestra.run.smithi035.stdout: "type": "rgw", 2019-12-09T13:54:56.638 INFO:teuthology.orchestra.run.smithi035.stdout: "mfa_ids": [] 2019-12-09T13:54:56.638 INFO:teuthology.orchestra.run.smithi035.stdout:} 2019-12-09T13:54:56.638 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T13:54:56.695 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.692+0000 7f491b75fec0 20 remove_watcher() i=0 2019-12-09T13:54:56.695 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.693+0000 7f491b75fec0 2 removed watcher, disabling cache 2019-12-09T13:54:56.789 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.787+0000 7f491b75fec0 20 remove_watcher() i=1 2019-12-09T13:54:56.840 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.838+0000 7f491b75fec0 20 remove_watcher() i=2 2019-12-09T13:54:56.898 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.896+0000 7f491b75fec0 20 remove_watcher() i=3 2019-12-09T13:54:56.951 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.948+0000 7f491b75fec0 20 remove_watcher() i=4 2019-12-09T13:54:56.992 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:56.989+0000 7f491b75fec0 20 remove_watcher() i=5 2019-12-09T13:54:57.024 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:57.022+0000 7f491b75fec0 20 remove_watcher() i=6 2019-12-09T13:54:57.059 INFO:teuthology.orchestra.run.smithi035.stderr:2019-12-09T13:54:57.057+0000 7f491b75fec0 20 remove_watcher() i=7 2019-12-09T13:54:57.067 INFO:tasks.swift:Configuring testswift... 2019-12-09T13:54:57.068 INFO:teuthology.orchestra.run.smithi035:> cd /home/ubuntu/cephtest/swift && ./bootstrap 2019-12-09T13:54:59.685 INFO:teuthology.orchestra.run.smithi035.stdout:New python executable in /home/ubuntu/cephtest/swift/virtualenv/bin/python 2019-12-09T13:55:01.133 INFO:teuthology.orchestra.run.smithi035.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T13:55:01.452 INFO:teuthology.orchestra.run.smithi035.stderr:Cache entry deserialization failed, entry ignored 2019-12-09T13:55:01.568 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting pip 2019-12-09T13:55:01.648 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-12-09T13:55:01.716 INFO:teuthology.orchestra.run.smithi035.stdout:Installing collected packages: pip 2019-12-09T13:55:01.716 INFO:teuthology.orchestra.run.smithi035.stdout: Found existing installation: pip 9.0.1 2019-12-09T13:55:01.785 INFO:teuthology.orchestra.run.smithi035.stdout: Uninstalling pip-9.0.1: 2019-12-09T13:55:01.834 INFO:teuthology.orchestra.run.smithi035.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T13:55:02.469 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully installed pip-19.3.1 2019-12-09T13:55:02.786 INFO:teuthology.orchestra.run.smithi035.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T13:55:03.244 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting setuptools==32.3.1 2019-12-09T13:55:03.247 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-12-09T13:55:03.288 INFO:teuthology.orchestra.run.smithi035.stdout:Installing collected packages: setuptools 2019-12-09T13:55:03.288 INFO:teuthology.orchestra.run.smithi035.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T13:55:03.305 INFO:teuthology.orchestra.run.smithi035.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T13:55:03.310 INFO:teuthology.orchestra.run.smithi035.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T13:55:03.437 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully installed setuptools-32.3.1 2019-12-09T13:55:03.926 INFO:teuthology.orchestra.run.smithi035.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T13:55:04.161 INFO:teuthology.orchestra.run.smithi035.stdout:Processing /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67/PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl 2019-12-09T13:55:04.238 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting nose>=1.0.0 2019-12-09T13:55:04.243 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-12-09T13:55:04.416 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting boto>=2.0b4 2019-12-09T13:55:04.425 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-12-09T13:55:04.578 INFO:teuthology.orchestra.run.smithi035.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-12-09T13:55:04.868 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting gevent>=0.14 2019-12-09T13:55:04.882 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T13:55:05.154 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting isodate>=0.4.4 2019-12-09T13:55:05.157 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl 2019-12-09T13:55:05.240 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting eventlet>=0.9.14 2019-12-09T13:55:05.288 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/ef/01/83454d11bac9015f34e93cf11efcae169390ddf2df97cb73ca86de6465ed/eventlet-0.25.1-py2.py3-none-any.whl (222kB) 2019-12-09T13:55:05.468 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting netifaces>=0.5 2019-12-09T13:55:05.477 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/7e/02/ad1a92a72620cc17d448fe4dbdfbdf8fe1487ee7bfd82bb48308712c2f3c/netifaces-0.10.9-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T13:55:05.589 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting xattr<0.7.0,>=0.6 2019-12-09T13:55:05.600 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/c5/80/b25d549ae4bf4f3e9635a331b759ffca2de4dd8a78dc5106d1ca92f5d08d/xattr-0.6.4.tar.gz 2019-12-09T13:55:06.124 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting simplejson>=2.1.3 2019-12-09T13:55:06.133 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/98/87/a7b98aa9256c8843f92878966dc3d8d914c14aad97e2c5ce4798d5743e07/simplejson-3.17.0.tar.gz (83kB) 2019-12-09T13:55:06.540 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting WebOb==1.0 2019-12-09T13:55:06.551 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/bc/eb/91b8dd8f3a8d3c3943bf27a2d322501ad8b227bf29f59e792c2eae092bc9/WebOb-1.0.tar.gz (210kB) 2019-12-09T13:55:07.006 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T13:55:07.010 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T13:55:07.067 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting six 2019-12-09T13:55:07.069 INFO:teuthology.orchestra.run.smithi035.stdout: Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-12-09T13:55:07.137 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting monotonic>=1.4 2019-12-09T13:55:07.141 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl 2019-12-09T13:55:07.225 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting dnspython>=1.15.0 2019-12-09T13:55:07.237 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/ec/d3/3aa0e7213ef72b8585747aa0e271a9523e713813b9a20177ebe1e939deb0/dnspython-1.16.0-py2.py3-none-any.whl (188kB) 2019-12-09T13:55:07.355 INFO:teuthology.orchestra.run.smithi035.stdout:Collecting enum34; python_version < "3.4" 2019-12-09T13:55:07.365 INFO:teuthology.orchestra.run.smithi035.stdout: Downloading https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl 2019-12-09T13:55:07.507 INFO:teuthology.orchestra.run.smithi035.stdout:Building wheels for collected packages: xattr, simplejson, WebOb 2019-12-09T13:55:07.507 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for xattr (setup.py): started 2019-12-09T13:55:07.976 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for xattr (setup.py): finished with status 'done' 2019-12-09T13:55:07.976 INFO:teuthology.orchestra.run.smithi035.stdout: Created wheel for xattr: filename=xattr-0.6.4-cp27-cp27mu-linux_x86_64.whl size=24116 sha256=a07b9c7d971cf3d3c4acfad5a64433bf8926bb8fc5469f9218d25ea6c572e433 2019-12-09T13:55:07.977 INFO:teuthology.orchestra.run.smithi035.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/cb/2c/40/ea48345a21b521cdacebf61db13e3bf0c42c77ec664e3fe9e5 2019-12-09T13:55:07.978 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for simplejson (setup.py): started 2019-12-09T13:55:09.041 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for simplejson (setup.py): finished with status 'done' 2019-12-09T13:55:09.043 INFO:teuthology.orchestra.run.smithi035.stdout: Created wheel for simplejson: filename=simplejson-3.17.0-cp27-cp27mu-linux_x86_64.whl size=113127 sha256=7f5b7a9d0a9a89fef473e4e2951f03a5602e76011714c01dd8a64ef21c57796e 2019-12-09T13:55:09.043 INFO:teuthology.orchestra.run.smithi035.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/86/c0/83/dcd0339abb2640544bb8e0938aab2d069cef55e5647ce6e097 2019-12-09T13:55:09.053 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for WebOb (setup.py): started 2019-12-09T13:55:09.311 INFO:teuthology.orchestra.run.smithi035.stdout: Building wheel for WebOb (setup.py): finished with status 'done' 2019-12-09T13:55:09.313 INFO:teuthology.orchestra.run.smithi035.stdout: Created wheel for WebOb: filename=WebOb-1.0-cp27-none-any.whl size=58478 sha256=03ba4ed2c2784bf371f256a3f8dfd0fd9290b6ee56b48e21380f3b379b9e7aea 2019-12-09T13:55:09.314 INFO:teuthology.orchestra.run.smithi035.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/30/41/c0/8ff9e679d72e05f355e6e2ec87d026131538c953a3b12496eb 2019-12-09T13:55:09.331 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully built xattr simplejson WebOb 2019-12-09T13:55:09.427 INFO:teuthology.orchestra.run.smithi035.stdout:Installing collected packages: PyYAML, nose, boto, bunch, greenlet, gevent, six, isodate, monotonic, dnspython, enum34, eventlet, netifaces, xattr, simplejson, WebOb 2019-12-09T13:55:10.904 INFO:teuthology.orchestra.run.smithi035.stdout:Successfully installed PyYAML-5.2 WebOb-1.0 boto-2.49.0 bunch-1.0.1 dnspython-1.16.0 enum34-1.1.6 eventlet-0.25.1 gevent-1.4.0 greenlet-0.4.15 isodate-0.6.0 monotonic-1.5 netifaces-0.10.9 nose-1.3.7 simplejson-3.17.0 six-1.13.0 xattr-0.6.4 2019-12-09T13:55:11.167 INFO:teuthology.orchestra.run.smithi035.stdout:running develop 2019-12-09T13:55:11.213 INFO:teuthology.orchestra.run.smithi035.stdout:running egg_info 2019-12-09T13:55:11.214 INFO:teuthology.orchestra.run.smithi035.stdout:creating swift.egg-info 2019-12-09T13:55:11.214 INFO:teuthology.orchestra.run.smithi035.stdout:writing swift.egg-info/PKG-INFO 2019-12-09T13:55:11.214 INFO:teuthology.orchestra.run.smithi035.stdout:writing top-level names to swift.egg-info/top_level.txt 2019-12-09T13:55:11.214 INFO:teuthology.orchestra.run.smithi035.stdout:writing dependency_links to swift.egg-info/dependency_links.txt 2019-12-09T13:55:11.219 INFO:teuthology.orchestra.run.smithi035.stdout:writing entry points to swift.egg-info/entry_points.txt 2019-12-09T13:55:11.220 INFO:teuthology.orchestra.run.smithi035.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T13:55:11.221 INFO:teuthology.orchestra.run.smithi035.stdout:reading manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T13:55:11.222 INFO:teuthology.orchestra.run.smithi035.stdout:reading manifest template 'MANIFEST.in' 2019-12-09T13:55:11.222 INFO:teuthology.orchestra.run.smithi035.stderr:warning: no files found matching 'ChangeLog' 2019-12-09T13:55:11.225 INFO:teuthology.orchestra.run.smithi035.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T13:55:11.226 INFO:teuthology.orchestra.run.smithi035.stdout:running build_ext 2019-12-09T13:55:11.226 INFO:teuthology.orchestra.run.smithi035.stdout:Creating /home/ubuntu/cephtest/swift/virtualenv/lib/python2.7/site-packages/swift.egg-link (link to .) 2019-12-09T13:55:11.226 INFO:teuthology.orchestra.run.smithi035.stdout:Adding swift 1.4.4 to easy-install.pth file 2019-12-09T13:55:11.244 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.244 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-account-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.246 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-account-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.247 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-account-reaper script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.247 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-account-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.249 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-account-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.249 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-container-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.250 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-container-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.250 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-container-sync script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.251 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-container-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.252 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-container-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.253 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-drive-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.254 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-get-nodes script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.255 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-init script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.256 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-object-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.257 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-object-expirer script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.258 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-object-info script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.259 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-object-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.260 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-object-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.261 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-object-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.262 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-proxy-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.265 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-ring-builder script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.267 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-stats-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.273 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-stats-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.275 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-dispersion-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.276 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-dispersion-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.277 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-bench script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.280 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-recon script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.281 INFO:teuthology.orchestra.run.smithi035.stdout:Installing swift-recon-cron script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:55:11.281 INFO:teuthology.orchestra.run.smithi035.stdout: 2019-12-09T13:55:11.281 INFO:teuthology.orchestra.run.smithi035.stdout:Installed /home/ubuntu/cephtest/swift 2019-12-09T13:55:11.282 INFO:teuthology.orchestra.run.smithi035.stdout:Processing dependencies for swift==1.4.4 2019-12-09T13:55:11.282 INFO:teuthology.orchestra.run.smithi035.stdout:Finished processing dependencies for swift==1.4.4 2019-12-09T13:55:11.293 INFO:teuthology.orchestra.run.smithi035:> cat > /home/ubuntu/cephtest/archive/testswift.client.0.conf 2019-12-09T13:55:11.343 INFO:teuthology.orchestra.run.smithi035:> SWIFT_TEST_CONFIG_FILE=/home/ubuntu/cephtest/archive/testswift.client.0.conf /home/ubuntu/cephtest/swift/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/swift/test/functional -v -a '!fails_on_rgw' 2019-12-09T13:55:39.139 INFO:teuthology.orchestra.run.smithi035.stderr:testAccountHead (test.functional.tests.TestAccount) ... ok 2019-12-09T13:55:51.470 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerListing (test.functional.tests.TestAccount) ... ok 2019-12-09T13:57:57.760 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:45:34.025 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:57.766 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:45:34.031 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:57.814 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:45:34.037 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:57.815 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:45:34.037 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:57.852 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:45:34.075 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:57.859 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:45:34.081 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:57.947 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:45:34.212 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:57.987 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:45:34.210 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:58.007 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:45:34.230 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:44.160 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:46:20.425 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:44.168 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:46:20.433 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:44.177 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:46:20.399 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:44.177 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:46:20.399 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:44.182 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:46:20.404 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:44.182 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:46:20.405 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:44.226 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:46:20.491 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:44.236 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:46:20.458 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:44.247 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:46:20.470 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T13:59:38.812 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerSerializedInfo (test.functional.tests.TestAccount) ... ok 2019-12-09T13:59:50.823 INFO:teuthology.orchestra.run.smithi035.stderr:testContainersOrderedByName (test.functional.tests.TestAccount) ... ok 2019-12-09T13:59:52.207 INFO:teuthology.orchestra.run.smithi035.stderr:testInvalidAuthToken (test.functional.tests.TestAccount) ... ok 2019-12-09T14:00:07.712 INFO:teuthology.orchestra.run.smithi035.stderr:testLastContainerMarker (test.functional.tests.TestAccount) ... ok 2019-12-09T14:00:23.850 INFO:teuthology.orchestra.run.smithi035.stderr:testListingLimit (test.functional.tests.TestAccount) ... ok 2019-12-09T14:01:15.021 INFO:teuthology.orchestra.run.smithi035.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccount) ... ok 2019-12-09T14:01:15.349 INFO:teuthology.orchestra.run.smithi035.stderr:testPUT (test.functional.tests.TestAccount) ... ok 2019-12-09T14:03:58.358 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:51:34.623 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:58.369 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:51:34.633 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:58.378 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:51:34.599 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:58.382 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:51:34.603 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:58.383 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:51:34.604 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:58.392 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:51:34.612 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:58.595 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:51:34.860 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:58.596 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:51:34.819 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:58.607 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:51:34.830 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:06:19.964 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:53:56.229 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:06:19.973 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:53:56.196 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:06:19.976 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:53:56.198 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:06:19.976 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:53:56.198 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:06:19.987 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:53:56.250 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:06:20.019 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:53:56.242 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:06:20.078 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:53:56.300 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:06:20.088 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:53:56.353 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:06:20.101 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:53:56.323 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:06:37.155 INFO:teuthology.orchestra.run.smithi035.stderr:testGetRequest (test.functional.tests.TestAccountNoContainers) ... ok 2019-12-09T14:06:38.889 INFO:teuthology.orchestra.run.smithi035.stderr:testGetRequest (test.functional.tests.TestAccountNoContainersUTF8) ... ok 2019-12-09T14:07:14.244 INFO:teuthology.orchestra.run.smithi035.stderr:testAccountHead (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:07:27.292 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerListing (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:09:10.605 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:56:46.870 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:10.613 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:56:46.878 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:10.616 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:56:46.839 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:10.620 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:56:46.843 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:10.625 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:56:46.848 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:10.629 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:56:46.852 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:10.740 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:04:56:47.005 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:10.782 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:04:56:47.005 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:10.795 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:04:56:47.017 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:12:00.890 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerSerializedInfo (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:12:13.475 INFO:teuthology.orchestra.run.smithi035.stderr:testContainersOrderedByName (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:12:15.138 INFO:teuthology.orchestra.run.smithi035.stderr:testInvalidAuthToken (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:12:28.758 INFO:teuthology.orchestra.run.smithi035.stderr:testLastContainerMarker (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:12:46.639 INFO:teuthology.orchestra.run.smithi035.stderr:testListingLimit (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:14:13.634 INFO:teuthology.orchestra.run.smithi035.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:14:13.687 INFO:teuthology.orchestra.run.smithi035.stderr:testPUT (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:14:21.663 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:01:57.928 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:21.667 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:01:57.932 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:21.680 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:01:57.903 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:21.680 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:01:57.903 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:21.682 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:01:57.905 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:21.685 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:01:57.908 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:21.878 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:01:58.100 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:21.881 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:01:58.104 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:21.933 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:01:58.198 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:42.537 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:02:18.801 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:42.541 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:02:18.806 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:42.556 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:02:18.778 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:42.571 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:02:18.836 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:42.595 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:02:18.818 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:42.612 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:02:18.835 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:42.622 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:02:18.844 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:42.636 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:02:18.858 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:42.646 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:02:18.868 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:24.003 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:07:00.268 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:24.023 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:07:00.282 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:24.072 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:07:00.294 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:24.073 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:07:00.296 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:24.088 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:07:00.311 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:24.114 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:07:00.337 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:24.250 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:07:00.515 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:24.305 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:07:00.528 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:24.342 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:07:00.564 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:32.278 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:07:08.543 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:32.289 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:07:08.555 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:32.295 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:07:08.518 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:32.299 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:07:08.522 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:32.324 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:07:08.546 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:32.329 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:07:08.552 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:32.330 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:07:08.595 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:32.330 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:07:08.553 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:32.378 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:07:08.601 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:33.609 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerExistenceCachingProblem (test.functional.tests.TestContainer) ... ok 2019-12-09T14:20:37.426 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerFileList (test.functional.tests.TestContainer) ... ok 2019-12-09T14:20:38.370 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerFileListOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T14:20:40.956 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerFileListWithLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T14:20:41.443 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerInfo (test.functional.tests.TestContainer) ... ok 2019-12-09T14:20:41.665 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerInfoOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T14:20:50.141 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerNameLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T14:20:55.503 INFO:teuthology.orchestra.run.smithi035.stderr:testCreate (test.functional.tests.TestContainer) ... ok 2019-12-09T14:21:03.877 INFO:teuthology.orchestra.run.smithi035.stderr:testDelete (test.functional.tests.TestContainer) ... ok 2019-12-09T14:21:04.731 INFO:teuthology.orchestra.run.smithi035.stderr:testDeleteOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T14:21:10.433 INFO:teuthology.orchestra.run.smithi035.stderr:testDeleteOnContainerWithFiles (test.functional.tests.TestContainer) ... ok 2019-12-09T14:21:11.485 INFO:teuthology.orchestra.run.smithi035.stderr:testFileCreateInContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T14:22:27.731 INFO:teuthology.orchestra.run.smithi035.stderr:testFileListingLimitMarkerPrefix (test.functional.tests.TestContainer) ... ok 2019-12-09T14:22:30.985 INFO:teuthology.orchestra.run.smithi035.stderr:testFileOrder (test.functional.tests.TestContainer) ... ok 2019-12-09T14:22:47.527 INFO:teuthology.orchestra.run.smithi035.stderr:testFileThenContainerDelete (test.functional.tests.TestContainer) ... ok 2019-12-09T14:22:54.629 INFO:teuthology.orchestra.run.smithi035.stderr:testLastFileMarker (test.functional.tests.TestContainer) ... ok 2019-12-09T14:23:05.028 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:10:41.292 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:05.074 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:10:41.339 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:05.095 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:10:41.317 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:05.147 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:10:41.370 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:05.150 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:10:41.372 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:05.153 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:10:41.374 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:05.180 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:10:41.446 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:05.252 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:10:41.474 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:05.326 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:10:41.549 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:25.384 INFO:teuthology.orchestra.run.smithi035.stderr:testMarkerLimitFileList (test.functional.tests.TestContainer) ... ok 2019-12-09T14:24:15.887 INFO:teuthology.orchestra.run.smithi035.stderr:testPrefixAndLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T14:24:15.981 INFO:teuthology.orchestra.run.smithi035.stderr:testTooLongName (test.functional.tests.TestContainer) ... ok 2019-12-09T14:24:43.703 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:12:19.967 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:43.708 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:12:19.973 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:43.721 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:12:19.940 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:43.725 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:12:19.948 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:43.731 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:12:19.954 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:43.740 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:12:19.963 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:43.827 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:12:20.092 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:43.849 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:12:20.072 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:43.863 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:12:20.086 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:25:09.760 INFO:teuthology.orchestra.run.smithi035.stderr:testUnorderedList (test.functional.tests.TestContainer) ... ok 2019-12-09T14:29:15.267 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerListing (test.functional.tests.TestContainerPaths) ... ok 2019-12-09T14:29:25.059 INFO:teuthology.orchestra.run.smithi035.stderr:testStructure (test.functional.tests.TestContainerPaths) ... ok 2019-12-09T14:29:40.656 INFO:teuthology.orchestra.run.smithi035.stderr:testTraverseContainer (test.functional.tests.TestContainerPaths) ... ok 2019-12-09T14:29:56.051 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:17:32.316 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:56.066 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:17:32.332 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:56.082 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:17:32.305 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:56.087 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:17:32.310 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:56.089 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:17:32.312 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:56.103 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:17:32.326 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:56.333 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:17:32.598 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:56.351 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:17:32.574 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:56.447 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:17:32.670 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:31:05.188 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerExistenceCachingProblem (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:31:08.110 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerFileList (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:31:09.142 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerFileListOnContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:31:11.943 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerFileListWithLimit (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:31:12.353 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerInfo (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:31:13.618 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerInfoOnContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:31:24.566 INFO:teuthology.orchestra.run.smithi035.stderr:testContainerNameLimit (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:31:27.381 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:19:03.646 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:31:27.437 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:19:03.702 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:31:27.450 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:19:03.673 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:31:27.471 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:19:03.694 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:31:27.472 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:19:03.695 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:31:27.473 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:19:03.696 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:31:27.500 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:19:03.766 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:31:27.531 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:19:03.754 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:31:27.531 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:19:03.754 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:31:29.851 INFO:teuthology.orchestra.run.smithi035.stderr:testCreate (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:31:39.263 INFO:teuthology.orchestra.run.smithi035.stderr:testDelete (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:31:39.320 INFO:teuthology.orchestra.run.smithi035.stderr:testDeleteOnContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:31:44.555 INFO:teuthology.orchestra.run.smithi035.stderr:testDeleteOnContainerWithFiles (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:31:44.909 INFO:teuthology.orchestra.run.smithi035.stderr:testFileCreateInContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:33:06.363 INFO:teuthology.orchestra.run.smithi035.stderr:testFileListingLimitMarkerPrefix (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:33:08.962 INFO:teuthology.orchestra.run.smithi035.stderr:testFileOrder (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:33:21.457 INFO:teuthology.orchestra.run.smithi035.stderr:testFileThenContainerDelete (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:33:28.916 INFO:teuthology.orchestra.run.smithi035.stderr:testLastFileMarker (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:34:05.943 INFO:teuthology.orchestra.run.smithi035.stderr:testMarkerLimitFileList (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:34:54.018 INFO:teuthology.orchestra.run.smithi035.stderr:testPrefixAndLimit (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:34:54.065 INFO:teuthology.orchestra.run.smithi035.stderr:testTooLongName (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:35:08.257 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:22:44.522 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:08.261 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:22:44.526 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:08.276 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:22:44.499 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:08.277 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:22:44.500 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:08.281 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:22:44.505 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:08.283 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:22:44.506 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:08.441 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:22:44.706 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:08.463 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:22:44.686 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:08.471 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:22:44.694 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:48.555 INFO:teuthology.orchestra.run.smithi035.stderr:testUnorderedList (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:39:46.192 INFO:teuthology.orchestra.run.smithi035.stderr:test_get_manifest (test.functional.tests.TestDlo) ... ok 2019-12-09T14:39:46.374 INFO:teuthology.orchestra.run.smithi035.stderr:test_get_manifest_document_itself (test.functional.tests.TestDlo) ... ok 2019-12-09T14:39:49.548 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:27:25.813 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:49.554 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:27:25.820 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:49.624 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:27:25.846 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:49.633 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:27:25.856 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:49.637 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:27:25.860 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:49.654 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:27:25.877 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:49.727 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:27:25.992 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:49.794 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:27:26.017 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:49.819 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:27:26.041 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:50.799 INFO:teuthology.orchestra.run.smithi035.stderr:test_get_range (test.functional.tests.TestDlo) ... ok 2019-12-09T14:39:51.658 INFO:teuthology.orchestra.run.smithi035.stderr:test_get_range_out_of_range (test.functional.tests.TestDlo) ... ok 2019-12-09T14:40:19.773 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:27:56.038 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:40:19.790 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:27:56.013 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:40:19.791 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:27:56.056 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:40:19.794 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:27:56.017 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:40:19.799 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:27:56.021 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:40:19.811 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:27:56.029 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:40:19.833 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:27:56.098 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:40:19.846 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:27:56.069 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:40:19.878 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:27:56.101 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:40:48.987 INFO:teuthology.orchestra.run.smithi035.stderr:test_get_manifest (test.functional.tests.TestDloUTF8) ... ok 2019-12-09T14:40:49.250 INFO:teuthology.orchestra.run.smithi035.stderr:test_get_manifest_document_itself (test.functional.tests.TestDloUTF8) ... ok 2019-12-09T14:40:56.328 INFO:teuthology.orchestra.run.smithi035.stderr:test_get_range (test.functional.tests.TestDloUTF8) ... ok 2019-12-09T14:40:57.210 INFO:teuthology.orchestra.run.smithi035.stderr:test_get_range_out_of_range (test.functional.tests.TestDloUTF8) ... ok 2019-12-09T14:41:26.777 INFO:teuthology.orchestra.run.smithi035.stderr:testBadHeaders (test.functional.tests.TestFile) ... ok 2019-12-09T14:41:26.816 INFO:teuthology.orchestra.run.smithi035.stderr:testBlankMetadataName (test.functional.tests.TestFile) ... ok 2019-12-09T14:45:31.067 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:33:07.332 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:31.082 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:33:07.305 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:31.084 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:33:07.349 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:31.091 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:33:07.314 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:31.098 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:33:07.321 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:31.106 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:33:07.329 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:31.301 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:33:07.566 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:31.307 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:33:07.530 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:31.374 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:33:07.597 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:40.941 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:38:17.206 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:40.949 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:38:17.214 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:40.955 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:38:17.177 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:40.959 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:38:17.182 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:40.959 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:38:17.182 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:40.968 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:38:17.191 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:41.146 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:38:17.368 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:41.146 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:38:17.369 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:41.205 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:38:17.470 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:53.165 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:43:29.430 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:53.167 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:43:29.390 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:53.175 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:43:29.398 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:53.175 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:43:29.398 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:53.181 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:43:29.403 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:53.182 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:43:29.448 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:53.372 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:43:29.595 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:53.384 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:43:29.607 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:53.447 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:43:29.712 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:56:34.166 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:44:10.431 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:56:34.229 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:44:10.487 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:56:34.291 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:44:10.556 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T14:56:34.293 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:44:10.516 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:56:34.299 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:44:10.522 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T14:56:34.310 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:44:10.533 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:56:34.313 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:44:10.536 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:56:34.353 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:44:10.576 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T14:56:34.357 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:44:10.580 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:25.460 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:48:01.725 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:25.488 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:48:01.753 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:25.520 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:48:01.742 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:25.563 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:48:01.785 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:25.612 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:48:01.835 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:25.624 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:48:01.847 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:25.716 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:48:01.939 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:25.770 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:48:02.035 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:25.827 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:48:02.050 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:05.388 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:48:41.653 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:05.398 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:48:41.662 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:05.406 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:48:41.628 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:05.408 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:48:41.631 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:05.409 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:48:41.632 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:05.413 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:48:41.635 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:05.465 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:48:41.687 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:05.481 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:48:41.704 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:05.492 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:48:41.757 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:51.201 INFO:teuthology.orchestra.run.smithi035.stderr:testChunkedPut (test.functional.tests.TestFile) ... ok 2019-12-09T15:01:58.311 INFO:teuthology.orchestra.run.smithi035.stderr:testContentTypeGuessing (test.functional.tests.TestFile) ... ok 2019-12-09T15:02:17.855 INFO:teuthology.orchestra.run.smithi035.stderr:testCopy (test.functional.tests.TestFile) ... ok 2019-12-09T15:02:25.044 INFO:teuthology.orchestra.run.smithi035.stderr:testCopy404s (test.functional.tests.TestFile) ... ok 2019-12-09T15:02:26.409 INFO:teuthology.orchestra.run.smithi035.stderr:testCopyDestinationSlashProblems (test.functional.tests.TestFile) ... ok 2019-12-09T15:02:43.610 INFO:teuthology.orchestra.run.smithi035.stderr:testCopyFromHeader (test.functional.tests.TestFile) ... ok 2019-12-09T15:02:48.757 INFO:teuthology.orchestra.run.smithi035.stderr:testCopyFromHeader404s (test.functional.tests.TestFile) ... ok 2019-12-09T15:02:50.949 INFO:teuthology.orchestra.run.smithi035.stderr:testCopyNoDestinationHeader (test.functional.tests.TestFile) ... ok 2019-12-09T15:02:54.642 INFO:teuthology.orchestra.run.smithi035.stderr:testDelete (test.functional.tests.TestFile) ... ok 2019-12-09T15:02:55.658 INFO:teuthology.orchestra.run.smithi035.stderr:testDeleteOfFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-12-09T15:03:00.785 INFO:teuthology.orchestra.run.smithi035.stderr:testDeleteThen404s (test.functional.tests.TestFile) ... ok 2019-12-09T15:03:02.175 INFO:teuthology.orchestra.run.smithi035.stderr:testEtagResponse (test.functional.tests.TestFile) ... ok 2019-12-09T15:03:02.673 INFO:teuthology.orchestra.run.smithi035.stderr:testEtagWayoff (test.functional.tests.TestFile) ... ok 2019-12-09T15:03:22.175 INFO:teuthology.orchestra.run.smithi035.stderr:testFileCreate (test.functional.tests.TestFile) ... ok 2019-12-09T15:03:23.584 INFO:teuthology.orchestra.run.smithi035.stderr:testGetContentType (test.functional.tests.TestFile) ... ok 2019-12-09T15:03:24.546 INFO:teuthology.orchestra.run.smithi035.stderr:testGetOnFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-12-09T15:03:26.055 INFO:teuthology.orchestra.run.smithi035.stderr:testHead (test.functional.tests.TestFile) ... ok 2019-12-09T15:03:26.599 INFO:teuthology.orchestra.run.smithi035.stderr:testHeadOnFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-12-09T15:03:46.832 INFO:teuthology.orchestra.run.smithi035.stderr:testMetadataOnPost (test.functional.tests.TestFile) ... ok 2019-12-09T15:04:02.646 INFO:teuthology.orchestra.run.smithi035.stderr:testMetadataOnPut (test.functional.tests.TestFile) ... ok 2019-12-09T15:04:07.797 INFO:teuthology.orchestra.run.smithi035.stderr:testNameLimit (test.functional.tests.TestFile) ... ok 2019-12-09T15:04:07.849 INFO:teuthology.orchestra.run.smithi035.stderr:testNoContentLengthForPut (test.functional.tests.TestFile) ... ok 2019-12-09T15:04:09.392 INFO:teuthology.orchestra.run.smithi035.stderr:testPostOnFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-12-09T15:04:32.128 INFO:teuthology.orchestra.run.smithi035.stderr:testRangedGets (test.functional.tests.TestFile) ... ok 2019-12-09T15:04:43.758 INFO:teuthology.orchestra.run.smithi035.stderr:testSerialization (test.functional.tests.TestFile) ... ok 2019-12-09T15:04:56.986 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:52:33.251 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:04:56.992 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:52:33.257 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:04:57.015 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:52:33.238 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:04:57.059 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:52:33.282 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:04:57.107 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:52:33.330 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:04:57.109 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:52:33.332 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:04:57.211 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:52:33.434 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:04:57.232 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:52:33.455 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:04:57.307 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:52:33.572 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:09.932 INFO:teuthology.orchestra.run.smithi035.stderr:testStackedOverwrite (test.functional.tests.TestFile) ... ok 2019-12-09T15:05:10.080 INFO:teuthology.orchestra.run.smithi035.stderr:testTooLongName (test.functional.tests.TestFile) ... ok 2019-12-09T15:05:13.020 INFO:teuthology.orchestra.run.smithi035.stderr:testZeroByteFile (test.functional.tests.TestFile) ... ok 2019-12-09T15:06:17.447 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:53:53.712 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:17.465 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:53:53.730 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:17.475 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:53:53.698 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:17.475 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:53:53.699 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:17.481 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:53:53.704 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:17.481 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:53:53.704 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:17.561 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:53:53.783 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:17.566 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:53:53.828 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:17.581 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:53:53.804 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:07:59.659 INFO:teuthology.orchestra.run.smithi035.stderr:testIfMatch (test.functional.tests.TestFileComparison) ... ok 2019-12-09T15:08:33.370 INFO:teuthology.orchestra.run.smithi035.stderr:testIfMatchAndUnmodified (test.functional.tests.TestFileComparison) ... ok 2019-12-09T15:08:51.721 INFO:teuthology.orchestra.run.smithi035.stderr:testIfModifiedSince (test.functional.tests.TestFileComparison) ... ok 2019-12-09T15:09:14.198 INFO:teuthology.orchestra.run.smithi035.stderr:testIfNoneMatch (test.functional.tests.TestFileComparison) ... ok 2019-12-09T15:09:34.308 INFO:teuthology.orchestra.run.smithi035.stderr:testIfUnmodifiedSince (test.functional.tests.TestFileComparison) ... ok 2019-12-09T15:10:53.087 INFO:teuthology.orchestra.run.smithi035.stderr:testIfMatch (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-12-09T15:11:22.375 INFO:teuthology.orchestra.run.smithi035.stderr:testIfMatchAndUnmodified (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-12-09T15:11:30.811 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:59:07.076 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:30.831 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:59:07.096 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:30.834 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:59:07.057 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:30.836 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:59:07.059 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:30.837 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:59:07.060 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:30.842 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:59:07.065 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:31.086 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:05:59:07.351 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:31.098 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:05:59:07.321 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:31.104 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:05:59:07.326 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:40.284 INFO:teuthology.orchestra.run.smithi035.stderr:testIfModifiedSince (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-12-09T15:11:58.624 INFO:teuthology.orchestra.run.smithi035.stderr:testIfNoneMatch (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-12-09T15:12:20.203 INFO:teuthology.orchestra.run.smithi035.stderr:testIfUnmodifiedSince (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-12-09T15:13:01.726 INFO:teuthology.orchestra.run.smithi035.stderr:testBadHeaders (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:13:01.847 INFO:teuthology.orchestra.run.smithi035.stderr:testBlankMetadataName (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:13:19.124 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:00:55.389 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:19.135 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:00:55.400 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:19.182 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:00:55.405 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:19.184 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:00:55.407 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:19.201 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:00:55.424 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:19.204 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:00:55.469 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:19.208 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:00:55.430 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:19.252 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:00:55.473 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:19.279 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:00:55.502 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:21.577 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:16:21.565+0000 1694f700 -1 osd.4 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.230826+0000 front 2019-12-09T15:15:41.204793+0000 (oldest deadline 2019-12-09T15:16:20.855571+0000) 2019-12-09T15:16:21.845 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:16:21.837+0000 1694f700 -1 osd.5 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.251692+0000 front 2019-12-09T15:15:41.457909+0000 (oldest deadline 2019-12-09T15:16:20.922050+0000) 2019-12-09T15:16:22.543 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:16:22.538+0000 1694f700 -1 osd.4 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.230826+0000 front 2019-12-09T15:15:41.204793+0000 (oldest deadline 2019-12-09T15:16:20.855571+0000) 2019-12-09T15:16:22.874 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:16:22.870+0000 1694f700 -1 osd.5 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.251692+0000 front 2019-12-09T15:15:41.457909+0000 (oldest deadline 2019-12-09T15:16:20.922050+0000) 2019-12-09T15:16:23.265 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:16:23.252+0000 1694f700 -1 osd.3 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:41.377353+0000 front 2019-12-09T15:15:41.300101+0000 (oldest deadline 2019-12-09T15:16:22.800067+0000) 2019-12-09T15:16:23.299 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:16:23.290+0000 1694f700 -1 osd.7 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:39.169258+0000 front 2019-12-09T15:15:39.173318+0000 (oldest deadline 2019-12-09T15:16:22.826724+0000) 2019-12-09T15:16:23.517 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:16:23.513+0000 1694f700 -1 osd.4 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.230826+0000 front 2019-12-09T15:15:41.204793+0000 (oldest deadline 2019-12-09T15:16:20.855571+0000) 2019-12-09T15:16:23.882 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:16:23.867+0000 1694f700 -1 osd.5 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.251692+0000 front 2019-12-09T15:15:41.457909+0000 (oldest deadline 2019-12-09T15:16:20.922050+0000) 2019-12-09T15:16:24.188 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:16:24.181+0000 1694f700 -1 osd.2 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.465111+0000 front 2019-12-09T15:15:40.745700+0000 (oldest deadline 2019-12-09T15:16:24.032978+0000) 2019-12-09T15:16:24.229 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:16:24.225+0000 1694f700 -1 osd.3 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:41.377353+0000 front 2019-12-09T15:15:41.300101+0000 (oldest deadline 2019-12-09T15:16:22.800067+0000) 2019-12-09T15:16:24.281 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:16:24.277+0000 1694f700 -1 osd.7 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:39.169258+0000 front 2019-12-09T15:15:39.173318+0000 (oldest deadline 2019-12-09T15:16:22.826724+0000) 2019-12-09T15:16:24.504 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:16:24.500+0000 1694f700 -1 osd.4 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.230826+0000 front 2019-12-09T15:15:41.204793+0000 (oldest deadline 2019-12-09T15:16:20.855571+0000) 2019-12-09T15:16:24.860 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:16:24.857+0000 1694f700 -1 osd.5 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.251692+0000 front 2019-12-09T15:15:41.457909+0000 (oldest deadline 2019-12-09T15:16:20.922050+0000) 2019-12-09T15:16:24.999 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:16:24.981+0000 1694f700 -1 osd.1 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.390956+0000 front 2019-12-09T15:15:40.558406+0000 (oldest deadline 2019-12-09T15:16:24.264748+0000) 2019-12-09T15:16:25.191 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:16:25.186+0000 1694f700 -1 osd.2 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.465111+0000 front 2019-12-09T15:15:40.745700+0000 (oldest deadline 2019-12-09T15:16:24.032978+0000) 2019-12-09T15:16:25.211 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:16:25.197+0000 1694f700 -1 osd.3 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:41.377353+0000 front 2019-12-09T15:15:41.300101+0000 (oldest deadline 2019-12-09T15:16:22.800067+0000) 2019-12-09T15:16:25.306 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:16:25.302+0000 1694f700 -1 osd.7 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:39.169258+0000 front 2019-12-09T15:15:39.173318+0000 (oldest deadline 2019-12-09T15:16:22.826724+0000) 2019-12-09T15:16:25.521 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:16:25.509+0000 1694f700 -1 osd.0 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.394404+0000 front 2019-12-09T15:15:40.720948+0000 (oldest deadline 2019-12-09T15:16:25.098696+0000) 2019-12-09T15:16:25.539 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:16:25.535+0000 1694f700 -1 osd.4 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.230826+0000 front 2019-12-09T15:15:41.204793+0000 (oldest deadline 2019-12-09T15:16:20.855571+0000) 2019-12-09T15:16:25.827 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:16:25.822+0000 1694f700 -1 osd.5 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.251692+0000 front 2019-12-09T15:15:41.457909+0000 (oldest deadline 2019-12-09T15:16:20.922050+0000) 2019-12-09T15:16:25.959 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:16:25.955+0000 1694f700 -1 osd.1 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.390956+0000 front 2019-12-09T15:15:40.558406+0000 (oldest deadline 2019-12-09T15:16:24.264748+0000) 2019-12-09T15:16:26.186 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:16:26.182+0000 1694f700 -1 osd.2 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.465111+0000 front 2019-12-09T15:15:40.745700+0000 (oldest deadline 2019-12-09T15:16:24.032978+0000) 2019-12-09T15:16:26.261 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:16:26.258+0000 1694f700 -1 osd.3 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:41.377353+0000 front 2019-12-09T15:15:41.300101+0000 (oldest deadline 2019-12-09T15:16:22.800067+0000) 2019-12-09T15:16:26.586 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:16:26.573+0000 1694f700 -1 osd.4 137 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:15:42.230826+0000 front 2019-12-09T15:15:41.204793+0000 (oldest deadline 2019-12-09T15:16:20.855571+0000) 2019-12-09T15:16:34.051 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:04:10.316 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:34.057 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:04:10.322 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:34.067 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:04:10.290 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:34.073 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:04:10.296 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:34.115 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:04:10.338 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:34.118 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:04:10.341 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:34.209 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:04:10.474 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:34.276 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:04:10.499 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:34.288 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:04:10.511 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:17:00.413 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:00.407+0000 3827c700 -1 osd.6 142 build_incremental_map_msg missing incremental map 142 2019-12-09T15:17:00.579 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:00.531+0000 35276700 -1 osd.6 142 build_incremental_map_msg missing incremental map 142 2019-12-09T15:17:00.617 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:00.608+0000 114ae700 -1 osd.6 142 build_incremental_map_msg missing incremental map 142 2019-12-09T15:17:00.653 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:00.632+0000 37a7b700 -1 osd.6 142 build_incremental_map_msg missing incremental map 142 2019-12-09T15:17:00.754 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:00.737+0000 34a75700 -1 osd.6 142 build_incremental_map_msg missing incremental map 142 2019-12-09T15:17:00.804 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:00.751+0000 36278700 -1 osd.6 142 build_incremental_map_msg missing incremental map 142 2019-12-09T15:17:00.854 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:00.837+0000 39a7f700 -1 osd.6 142 build_incremental_map_msg missing incremental map 142 2019-12-09T15:17:00.857 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:00.845+0000 3b282700 -1 osd.6 142 build_incremental_map_msg missing incremental map 142 2019-12-09T15:17:29.963 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:29.953+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:30.935 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:30.932+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:31.967 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:31.964+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:32.179 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:32.171+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:33.030 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:33.019+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:33.215 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:33.212+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:34.030 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:34.027+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:34.192 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:34.189+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:35.055 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:35.052+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:35.162 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:35.158+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:36.039 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:36.035+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:36.157 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:36.154+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:37.081 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:37.078+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:37.116 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:37.111+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:38.101 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:38.098+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:38.169 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:38.164+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:39.134 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:39.130+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:39.140 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:39.136+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:40.117 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:40.114+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:40.182 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:40.179+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:41.086 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:41.083+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:41.234 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:41.231+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:42.084 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:42.081+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:42.253 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:42.249+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:43.113 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:43.109+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:43.272 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:43.269+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:44.151 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:44.148+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:44.252 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:44.248+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:45.137 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:45.134+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:45.290 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:45.286+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:46.101 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:46.097+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:46.265 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:46.261+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:47.148 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:47.144+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:47.302 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:47.297+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:48.179 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:48.176+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:48.280 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:48.275+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:49.162 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:49.158+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:49.315 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:49.311+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:50.169 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:50.164+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:50.362 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:50.356+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:51.175 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:51.171+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:51.388 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:51.382+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:52.186 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:52.182+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:52.376 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:52.371+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:53.180 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:53.176+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:53.338 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:53.333+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:54.151 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:54.147+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:54.389 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:54.385+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:55.162 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:55.158+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:55.380 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:55.375+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:56.121 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:56.116+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:56.351 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:56.346+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:57.087 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:57.081+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:57.339 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:57.334+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:58.073 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:58.068+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:58.300 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:58.295+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:17:59.089 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:17:59.086+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:17:59.339 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:17:59.334+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:00.086 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:00.081+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:00.369 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:00.365+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:01.137 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:01.134+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:01.382 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:01.377+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:02.132 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:02.128+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:02.387 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:02.383+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:03.096 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:03.092+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:03.362 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:03.355+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:04.145 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:04.143+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:04.409 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:04.404+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:05.187 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:05.184+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:05.415 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:05.411+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:06.256 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:06.242+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:06.461 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:06.457+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:07.220 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:07.217+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:07.474 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:07.469+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:08.265 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:08.261+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:08.493 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:08.488+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:09.291 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:09.286+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:09.431 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:18:09.424+0000 1694f700 -1 osd.4 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:26.048617+0000 front 2019-12-09T15:17:25.925493+0000 (oldest deadline 2019-12-09T15:18:08.728796+0000) 2019-12-09T15:18:09.460 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:09.456+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:09.481 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:18:09.474+0000 1694f700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:24.925722+0000 front 2019-12-09T15:17:25.221666+0000 (oldest deadline 2019-12-09T15:18:08.827867+0000) 2019-12-09T15:18:09.502 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:18:09.496+0000 1694f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:23.879839+0000 front 2019-12-09T15:17:24.013191+0000 (oldest deadline 2019-12-09T15:18:09.129871+0000) 2019-12-09T15:18:10.216 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:18:10.210+0000 1694f700 -1 osd.2 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:26.085489+0000 front 2019-12-09T15:17:26.049432+0000 (oldest deadline 2019-12-09T15:18:09.992844+0000) 2019-12-09T15:18:10.320 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:10.316+0000 1694f700 -1 osd.0 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:26.272458+0000 front 2019-12-09T15:17:27.099140+0000 (oldest deadline 2019-12-09T15:18:09.588890+0000) 2019-12-09T15:18:10.324 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:10.321+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:10.414 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:18:10.408+0000 1694f700 -1 osd.4 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:26.048617+0000 front 2019-12-09T15:17:25.925493+0000 (oldest deadline 2019-12-09T15:18:08.728796+0000) 2019-12-09T15:18:10.483 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:18:10.476+0000 1694f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:23.879839+0000 front 2019-12-09T15:17:24.013191+0000 (oldest deadline 2019-12-09T15:18:09.129871+0000) 2019-12-09T15:18:10.483 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:18:10.477+0000 1694f700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:24.925722+0000 front 2019-12-09T15:17:25.221666+0000 (oldest deadline 2019-12-09T15:18:08.827867+0000) 2019-12-09T15:18:10.503 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:10.499+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:11.029 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:18:11.025+0000 1694f700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:26.988599+0000 front 2019-12-09T15:17:27.239766+0000 (oldest deadline 2019-12-09T15:18:10.276168+0000) 2019-12-09T15:18:11.264 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:18:11.259+0000 1694f700 -1 osd.2 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:26.085489+0000 front 2019-12-09T15:17:26.049432+0000 (oldest deadline 2019-12-09T15:18:09.992844+0000) 2019-12-09T15:18:11.350 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:11.339+0000 1694f700 -1 osd.0 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:26.272458+0000 front 2019-12-09T15:17:27.099140+0000 (oldest deadline 2019-12-09T15:18:09.588890+0000) 2019-12-09T15:18:11.357 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:11.347+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:11.451 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:18:11.443+0000 1694f700 -1 osd.4 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:26.048617+0000 front 2019-12-09T15:17:25.925493+0000 (oldest deadline 2019-12-09T15:18:08.728796+0000) 2019-12-09T15:18:11.473 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:18:11.466+0000 1694f700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:23.879839+0000 front 2019-12-09T15:17:24.013191+0000 (oldest deadline 2019-12-09T15:18:09.129871+0000) 2019-12-09T15:18:11.486 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:18:11.480+0000 1694f700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:24.925722+0000 front 2019-12-09T15:17:25.221666+0000 (oldest deadline 2019-12-09T15:18:08.827867+0000) 2019-12-09T15:18:11.494 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:11.489+0000 1694f700 -1 osd.6 143 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:12.032 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:18:12.028+0000 1694f700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:26.988599+0000 front 2019-12-09T15:17:27.239766+0000 (oldest deadline 2019-12-09T15:18:10.276168+0000) 2019-12-09T15:18:12.230 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:18:12.225+0000 1694f700 -1 osd.2 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:26.085489+0000 front 2019-12-09T15:17:26.049432+0000 (oldest deadline 2019-12-09T15:18:09.992844+0000) 2019-12-09T15:18:12.496 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:18:12.491+0000 1694f700 -1 osd.4 143 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:17:26.048617+0000 front 2019-12-09T15:17:25.925493+0000 (oldest deadline 2019-12-09T15:18:08.728796+0000) 2019-12-09T15:18:12.691 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:12.571+0000 1694f700 -1 osd.0 144 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:12.791 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:18:12.739+0000 1694f700 -1 osd.6 144 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-12-09T15:18:13.837 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:18:13.824+0000 1694f700 -1 osd.0 145 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:111306 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821536544 gen 68] snapc 0=[] ondisk+write+known_if_redirected e140) 2019-12-09T15:18:14.702 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T15:18:14.630+0000 20e25700 -1 RGWWatcher::handle_error cookie 821536544 err (107) Transport endpoint is not connected 2019-12-09T15:19:51.653 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:19:51.647+0000 1694f700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:10.497132+0000 front 2019-12-09T15:19:10.647914+0000 (oldest deadline 2019-12-09T15:19:51.353499+0000) 2019-12-09T15:19:51.907 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:19:51.902+0000 1694f700 -1 osd.2 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:08.605408+0000 front 2019-12-09T15:19:09.668364+0000 (oldest deadline 2019-12-09T15:19:51.612745+0000) 2019-12-09T15:19:52.642 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:19:52.639+0000 1694f700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:09.941562+0000 front 2019-12-09T15:19:09.352680+0000 (oldest deadline 2019-12-09T15:19:52.076321+0000) 2019-12-09T15:19:52.652 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:19:52.646+0000 1694f700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:10.497132+0000 front 2019-12-09T15:19:10.647914+0000 (oldest deadline 2019-12-09T15:19:51.353499+0000) 2019-12-09T15:19:52.933 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:19:52.929+0000 1694f700 -1 osd.2 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:08.605408+0000 front 2019-12-09T15:19:09.668364+0000 (oldest deadline 2019-12-09T15:19:51.612745+0000) 2019-12-09T15:19:53.155 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:19:53.149+0000 1694f700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:09.766118+0000 front 2019-12-09T15:19:09.765176+0000 (oldest deadline 2019-12-09T15:19:52.275003+0000) 2019-12-09T15:19:53.631 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:19:53.626+0000 1694f700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:10.497132+0000 front 2019-12-09T15:19:10.647914+0000 (oldest deadline 2019-12-09T15:19:51.353499+0000) 2019-12-09T15:19:53.640 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:19:53.633+0000 1694f700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:09.941562+0000 front 2019-12-09T15:19:09.352680+0000 (oldest deadline 2019-12-09T15:19:52.076321+0000) 2019-12-09T15:19:53.937 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:19:53.933+0000 1694f700 -1 osd.2 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:08.605408+0000 front 2019-12-09T15:19:09.668364+0000 (oldest deadline 2019-12-09T15:19:51.612745+0000) 2019-12-09T15:19:54.116 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:19:54.109+0000 1694f700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:09.766118+0000 front 2019-12-09T15:19:09.765176+0000 (oldest deadline 2019-12-09T15:19:52.275003+0000) 2019-12-09T15:19:54.232 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:19:54.220+0000 1694f700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:09.679064+0000 front 2019-12-09T15:19:09.932587+0000 (oldest deadline 2019-12-09T15:19:54.217151+0000) 2019-12-09T15:19:54.324 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:19:54.319+0000 1694f700 -1 osd.1 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:09.844380+0000 front 2019-12-09T15:19:09.354379+0000 (oldest deadline 2019-12-09T15:19:53.899567+0000) 2019-12-09T15:19:54.620 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:19:54.615+0000 1694f700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:09.941562+0000 front 2019-12-09T15:19:09.352680+0000 (oldest deadline 2019-12-09T15:19:52.076321+0000) 2019-12-09T15:19:54.622 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:19:54.618+0000 1694f700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:10.497132+0000 front 2019-12-09T15:19:10.647914+0000 (oldest deadline 2019-12-09T15:19:51.353499+0000) 2019-12-09T15:19:54.956 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:19:54.952+0000 1694f700 -1 osd.2 149 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:19:08.605408+0000 front 2019-12-09T15:19:09.668364+0000 (oldest deadline 2019-12-09T15:19:51.612745+0000) 2019-12-09T15:20:37.346 INFO:teuthology.orchestra.run.smithi035.stderr:testChunkedPut (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:20:44.194 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:08:20.459 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:44.202 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:08:20.467 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:44.208 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:08:20.430 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:44.209 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:08:20.432 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:44.214 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:08:20.437 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:44.215 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:08:20.438 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:44.477 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:08:20.742 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:44.534 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:08:20.756 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:44.606 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:08:20.829 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:49.806 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:20:49.803+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:20:50.813 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:20:50.810+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:20:51.834 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:20:51.831+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:20:52.878 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:20:52.874+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:20:53.891 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:20:53.887+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:20:54.906 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:20:54.903+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:20:55.002 INFO:teuthology.orchestra.run.smithi035.stderr:testContentTypeGuessing (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:20:55.917 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:20:55.914+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:20:56.937 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:20:56.933+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:20:57.891 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:20:57.888+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:20:58.865 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:20:58.862+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:20:59.820 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:20:59.817+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:00.808 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:00.800+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:01.814 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:01.810+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:02.622 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:21:02.594+0000 3aa81700 -1 osd.1 151 build_incremental_map_msg missing incremental map 149 2019-12-09T15:21:02.794 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:02.791+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:03.811 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:03.808+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:04.862 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:04.859+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:05.861 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:05.857+0000 1694f700 -1 osd.0 151 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:06.997 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:06.859+0000 1694f700 -1 osd.0 152 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:07.895 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:07.890+0000 1694f700 -1 osd.0 152 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:09.056 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:09.052+0000 1694f700 -1 osd.0 153 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:10.061 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:10.047+0000 1694f700 -1 osd.0 154 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:11.181 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:11.143+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:12.187 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:12.185+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:13.172 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:13.168+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:14.172 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:14.168+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:14.470 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:21:14.461+0000 3827c700 -1 osd.6 155 build_incremental_map_msg missing incremental map 142 2019-12-09T15:21:15.165 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:15.162+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:16.133 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:16.128+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:17.171 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:17.168+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:18.128 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:18.125+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:19.094 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:19.088+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:20.087 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:20.084+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:21.130 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:21.126+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:22.102 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:22.098+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:23.147 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:23.143+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:24.150 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:24.146+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:25.179 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:25.175+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:26.210 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:26.206+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:27.165 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:27.161+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:28.204 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:28.199+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:29.197 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:29.193+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:30.178 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:30.174+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:31.230 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:31.226+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:32.277 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:32.273+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:33.290 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:33.283+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:34.272 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:34.268+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:35.288 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:35.284+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:36.326 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:36.321+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:37.313 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:37.308+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:38.061 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:21:38.046+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:55.039841+0000 front 2019-12-09T15:20:54.341473+0000 (oldest deadline 2019-12-09T15:21:37.837790+0000) 2019-12-09T15:21:38.315 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:21:38.311+0000 1694f700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:53.230451+0000 front 2019-12-09T15:20:53.088391+0000 (oldest deadline 2019-12-09T15:21:37.879313+0000) 2019-12-09T15:21:38.326 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:38.322+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:38.343 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:21:38.334+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:53.189817+0000 front 2019-12-09T15:20:52.608336+0000 (oldest deadline 2019-12-09T15:21:37.815064+0000) 2019-12-09T15:21:38.728 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:21:38.722+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:55.740814+0000 front 2019-12-09T15:20:55.652597+0000 (oldest deadline 2019-12-09T15:21:38.532227+0000) 2019-12-09T15:21:39.075 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:21:39.069+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:55.039841+0000 front 2019-12-09T15:20:54.341473+0000 (oldest deadline 2019-12-09T15:21:37.837790+0000) 2019-12-09T15:21:39.316 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:21:39.311+0000 1694f700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:53.230451+0000 front 2019-12-09T15:20:53.088391+0000 (oldest deadline 2019-12-09T15:21:37.879313+0000) 2019-12-09T15:21:39.332 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:21:39.327+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:53.189817+0000 front 2019-12-09T15:20:52.608336+0000 (oldest deadline 2019-12-09T15:21:37.815064+0000) 2019-12-09T15:21:39.357 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:39.345+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:56.476365+0000 front 2019-12-09T15:20:56.739049+0000 (oldest deadline 2019-12-09T15:21:38.777122+0000) 2019-12-09T15:21:39.358 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:39.352+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:39.735 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:21:39.729+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:55.740814+0000 front 2019-12-09T15:20:55.652597+0000 (oldest deadline 2019-12-09T15:21:38.532227+0000) 2019-12-09T15:21:40.048 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:21:40.043+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:55.039841+0000 front 2019-12-09T15:20:54.341473+0000 (oldest deadline 2019-12-09T15:21:37.837790+0000) 2019-12-09T15:21:40.274 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:21:40.270+0000 1694f700 -1 osd.6 155 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:111628 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e153) 2019-12-09T15:21:40.325 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:40.318+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:56.476365+0000 front 2019-12-09T15:20:56.739049+0000 (oldest deadline 2019-12-09T15:21:38.777122+0000) 2019-12-09T15:21:40.327 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:40.322+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:40.352 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:21:40.347+0000 1694f700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:53.230451+0000 front 2019-12-09T15:20:53.088391+0000 (oldest deadline 2019-12-09T15:21:37.879313+0000) 2019-12-09T15:21:40.354 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:21:40.348+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:54.240498+0000 front 2019-12-09T15:20:54.183062+0000 (oldest deadline 2019-12-09T15:21:39.857124+0000) 2019-12-09T15:21:40.372 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:21:40.366+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:53.189817+0000 front 2019-12-09T15:20:52.608336+0000 (oldest deadline 2019-12-09T15:21:37.815064+0000) 2019-12-09T15:21:40.769 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:21:40.761+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:55.740814+0000 front 2019-12-09T15:20:55.652597+0000 (oldest deadline 2019-12-09T15:21:38.532227+0000) 2019-12-09T15:21:41.032 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:21:41.026+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:55.039841+0000 front 2019-12-09T15:20:54.341473+0000 (oldest deadline 2019-12-09T15:21:37.837790+0000) 2019-12-09T15:21:41.279 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:21:41.274+0000 1694f700 -1 osd.6 155 get_health_metrics reporting 19 slow ops, oldest is osd_op(client.4355.0:111628 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e153) 2019-12-09T15:21:41.304 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:41.299+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:56.476365+0000 front 2019-12-09T15:20:56.739049+0000 (oldest deadline 2019-12-09T15:21:38.777122+0000) 2019-12-09T15:21:41.306 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:41.301+0000 1694f700 -1 osd.0 155 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:41.367 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:21:41.363+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:54.240498+0000 front 2019-12-09T15:20:54.183062+0000 (oldest deadline 2019-12-09T15:21:39.857124+0000) 2019-12-09T15:21:41.388 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:21:41.383+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:53.189817+0000 front 2019-12-09T15:20:52.608336+0000 (oldest deadline 2019-12-09T15:21:37.815064+0000) 2019-12-09T15:21:41.389 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:21:41.383+0000 1694f700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:53.230451+0000 front 2019-12-09T15:20:53.088391+0000 (oldest deadline 2019-12-09T15:21:37.879313+0000) 2019-12-09T15:21:41.544 INFO:teuthology.orchestra.run.smithi035.stderr:testCopy (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:21:41.684 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:09:17.949 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:41.706 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:09:17.929 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:41.713 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:09:17.931 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:41.713 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:09:17.932 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:41.722 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:09:17.987 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:41.752 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:09:17.974 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:41.803 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:21:41.797+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:20:55.740814+0000 front 2019-12-09T15:20:55.652597+0000 (oldest deadline 2019-12-09T15:21:38.532227+0000) 2019-12-09T15:21:41.826 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:09:18.091 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:41.865 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:09:18.088 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:41.880 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:09:18.100 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:42.368 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:42.337+0000 1694f700 -1 osd.0 156 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:43.382 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:21:43.354+0000 1694f700 -1 osd.0 157 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:111648 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399580928 gen 1] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T15:21:44.752 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T15:21:44.698+0000 20e25700 -1 RGWWatcher::handle_error cookie 399580928 err (107) Transport endpoint is not connected 2019-12-09T15:22:17.209 INFO:teuthology.orchestra.run.smithi035.stderr:testCopy404s (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:22:18.757 INFO:teuthology.orchestra.run.smithi035.stderr:testCopyDestinationSlashProblems (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:22:59.358 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:22:59.349+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:00.381 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:00.377+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:01.389 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:01.384+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:02.440 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:02.436+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:03.419 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:03.414+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:04.463 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:04.459+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:05.495 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:05.490+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:06.489 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:06.484+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:07.489 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:07.484+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:08.476 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:08.471+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:09.464 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:09.460+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:10.459 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:10.454+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:11.446 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:11.441+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:12.402 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:12.398+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:13.402 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:13.397+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:14.430 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:14.426+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:15.464 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:15.459+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:16.492 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:16.487+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:17.498 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:17.494+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:18.470 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:18.466+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:19.510 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:19.505+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:20.507 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:20.503+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:21.554 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:21.550+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:22.567 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:22.563+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:23.505 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:23.499+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:23.601 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:23.596+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:23.880 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:23.875+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:23.989 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:23.984+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:24.512 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:24.507+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:24.623 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:24.618+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:24.774 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:24.770+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:24.915 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:24.910+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:24.982 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:24.978+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:25.510 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:25.505+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:25.618 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:25.613+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:25.624 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:25.619+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:25.737 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:25.731+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:25.918 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:25.913+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:25.940 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:25.936+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:25.970 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:25.964+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:26.534 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:26.528+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:26.616 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:26.611+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:26.617 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:26.612+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:26.734 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:26.730+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:26.901 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:26.895+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:26.910 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:26.907+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:26.962 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:26.957+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:26.981 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:26.976+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:27.535 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:27.530+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:27.638 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:27.631+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:27.639 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:27.632+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:27.725 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:27.721+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:27.888 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:27.883+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:27.908 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:27.902+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:28.005 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:28.001+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:28.011 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:28.005+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:28.579 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:28.574+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:28.661 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:28.656+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:28.664 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:28.660+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:28.778 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:28.775+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:28.890 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:28.883+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:28.912 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:28.907+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:28.986 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:28.983+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:29.001 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:28.996+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:29.545 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:29.540+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:29.711 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:29.705+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:29.711 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:29.706+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:29.781 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:29.777+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:29.907 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:29.904+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:29.916 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:29.911+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:30.011 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:30.007+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:30.034 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:30.030+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:30.557 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:30.552+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:30.735 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:30.729+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:30.736 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:30.731+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:30.820 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:30.817+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:30.876 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:30.871+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:30.929 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:30.923+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:31.051 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:31.045+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:31.063 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:31.056+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:31.581 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:31.575+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:31.696 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:31.691+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:31.700 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:31.695+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:31.786 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:31.781+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:31.861 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:31.855+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:31.983 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:31.977+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:32.074 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:32.068+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:32.110 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:32.105+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:32.572 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:32.567+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:32.667 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:32.660+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:32.668 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:32.662+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:32.760 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:32.753+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:32.816 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:32.811+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:33.013 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:33.009+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:33.068 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:33.062+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:33.120 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:33.112+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:33.628 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:33.622+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:33.718 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:33.712+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:33.718 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:33.713+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:33.813 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:33.807+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:33.847 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:33.842+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:34.048 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:34.041+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:34.064 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:34.059+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:34.175 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:34.165+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:34.598 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:34.594+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:34.748 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:34.744+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:34.750 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:34.747+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:34.854 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:34.849+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:34.872 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:34.869+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:35.025 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:35.020+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:35.093 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:35.088+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:35.203 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:35.199+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:35.569 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:35.563+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:35.713 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:35.708+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:35.713 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:35.709+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:35.813 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:35.809+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:35.887 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:35.882+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:36.056 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:36.050+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:36.083 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:36.076+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:36.209 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:36.204+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:36.574 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:36.568+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:36.674 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:36.669+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:36.676 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:36.672+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:36.796 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:36.792+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:36.933 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:36.927+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:37.084 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:37.079+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:37.111 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:37.107+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:37.265 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:37.258+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:37.573 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:37.567+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:37.662 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:37.658+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:37.664 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:37.660+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:37.796 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:37.792+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:37.951 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:37.948+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:38.079 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:38.072+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:38.112 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:38.108+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:38.256 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:38.251+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:38.588 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:38.582+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:38.699 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:38.694+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:38.700 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:38.695+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:38.832 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:38.828+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:38.934 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:38.929+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:39.036 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:39.030+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:39.167 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:39.162+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:39.243 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:39.239+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:39.628 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:39.623+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:39.654 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:39.649+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:39.657 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:39.653+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:39.874 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:39.870+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:39.967 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:39.963+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:40.068 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:40.063+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:40.139 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:40.135+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:40.207 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:40.202+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:40.662 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:40.657+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:40.706 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:40.660+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:40.707 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:40.668+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:40.861 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:40.856+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:40.973 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:40.968+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:41.128 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:41.058+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:41.128 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:41.119+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:41.160 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:41.156+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:41.647 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:23:41.641+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.610911+0000 front 2019-12-09T15:22:41.786945+0000 (oldest deadline 2019-12-09T15:23:23.043734+0000) 2019-12-09T15:23:41.657 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:41.651+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:43.418436+0000 front 2019-12-09T15:22:43.401419+0000 (oldest deadline 2019-12-09T15:23:25.073056+0000) 2019-12-09T15:23:41.658 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:41.653+0000 1694f700 -1 osd.7 161 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs out=1132b,stat out=16b,read 0~25206] snapc 0=[] ondisk+read+known_if_redirected e161) 2019-12-09T15:23:41.852 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:23:41.847+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:39.303876+0000 front 2019-12-09T15:22:40.531398+0000 (oldest deadline 2019-12-09T15:23:24.471352+0000) 2019-12-09T15:23:41.972 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:23:41.969+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.833859+0000 front 2019-12-09T15:22:41.806632+0000 (oldest deadline 2019-12-09T15:23:26.859197+0000) 2019-12-09T15:23:42.040 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:23:42.035+0000 1694f700 -1 osd.5 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.223821+0000 front 2019-12-09T15:22:43.080233+0000 (oldest deadline 2019-12-09T15:23:22.995038+0000) 2019-12-09T15:23:42.100 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:23:42.096+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:41.796658+0000 front 2019-12-09T15:22:40.223244+0000 (oldest deadline 2019-12-09T15:23:25.466905+0000) 2019-12-09T15:23:42.174 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:23:42.169+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:22:40.618796+0000 front 2019-12-09T15:22:41.860375+0000 (oldest deadline 2019-12-09T15:23:23.429148+0000) 2019-12-09T15:23:42.945 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:42.919+0000 1694f700 -1 osd.7 162 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs,stat,read 0~4194304] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e161) 2019-12-09T15:23:44.199 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:23:44.053+0000 1694f700 -1 osd.7 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112077 2.1ds0 2:bb7676c3:::d0d8af73-443a-4cb3-a612-7fc7fb9ae85c.4361.658_%e1%a0%81%e8%94%8b%eb%8c%8d%ed%88%90%eb%8c%8e%e3%9c%86%ce%a9%ce%a9%ef%84%90%e0%a4%80%e9%90%8c%e3%9c%86%eb%8c%8e%e0%a4%83%e0%a4%82:head [getxattrs,stat,read 0~4194304] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e161) 2019-12-09T15:24:22.615 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:11:58.880 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:22.626 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:11:58.890 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:22.635 INFO:teuthology.orchestra.run.smithi035.stderr:testCopyFromHeader (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:24:22.683 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:11:58.905 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:22.697 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:11:58.920 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:22.700 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:11:58.923 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:22.714 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:11:58.936 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:22.863 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:11:59.127 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:22.921 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:11:59.143 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:22.933 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:11:59.155 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:32.735 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:12:09.000 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:32.747 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:12:08.970 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:32.749 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:12:08.972 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:32.752 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:12:09.017 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:32.753 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:12:08.976 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:32.754 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:12:08.977 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:32.785 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:12:09.008 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:32.790 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:12:09.012 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:32.790 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:12:09.056 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:38.253 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:38.250+0000 1694f700 -1 osd.0 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:39.259 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:39.256+0000 1694f700 -1 osd.0 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:39.427 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:39.423+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:40.127 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:40.122+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:40.282 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:40.279+0000 1694f700 -1 osd.0 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:40.530 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:40.508+0000 1694f700 -1 osd.2 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:41.119 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:41.114+0000 1694f700 -1 osd.7 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:41.267 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:41.261+0000 1694f700 -1 osd.0 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:41.268 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:41.263+0000 1694f700 -1 osd.0 164 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:41.346 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:41.340+0000 1694f700 -1 osd.5 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:41.412 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:24:41.407+0000 1694f700 -1 osd.4 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.112127+0000 front 2019-12-09T15:23:56.096301+0000 (oldest deadline 2019-12-09T15:24:41.125864+0000) 2019-12-09T15:24:41.506 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:41.500+0000 1694f700 -1 osd.3 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:41.570 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:41.567+0000 1694f700 -1 osd.2 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:42.166 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:42.159+0000 1694f700 -1 osd.7 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:42.258 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:42.254+0000 1694f700 -1 osd.0 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:42.260 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:42.257+0000 1694f700 -1 osd.0 164 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:42.389 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:42.385+0000 1694f700 -1 osd.5 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:42.441 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:24:42.435+0000 1694f700 -1 osd.4 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.112127+0000 front 2019-12-09T15:23:56.096301+0000 (oldest deadline 2019-12-09T15:24:41.125864+0000) 2019-12-09T15:24:42.529 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:42.525+0000 1694f700 -1 osd.3 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:42.590 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:42.584+0000 1694f700 -1 osd.2 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:43.172 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:43.166+0000 1694f700 -1 osd.7 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:43.307 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:43.301+0000 1694f700 -1 osd.0 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:43.309 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:43.304+0000 1694f700 -1 osd.0 164 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:43.403 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:43.398+0000 1694f700 -1 osd.5 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:43.433 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:24:43.427+0000 1694f700 -1 osd.4 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.112127+0000 front 2019-12-09T15:23:56.096301+0000 (oldest deadline 2019-12-09T15:24:41.125864+0000) 2019-12-09T15:24:43.502 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:43.498+0000 1694f700 -1 osd.3 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:43.582 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:43.579+0000 1694f700 -1 osd.2 164 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:44.320 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:44.313+0000 1694f700 -1 osd.0 165 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:44.322 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:44.315+0000 1694f700 -1 osd.7 165 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:44.324 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:44.321+0000 1694f700 -1 osd.0 165 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:44.503 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:44.498+0000 1694f700 -1 osd.3 165 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:44.536 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:44.451+0000 3aa81700 -1 osd.2 165 build_incremental_map_msg missing incremental map 165 2019-12-09T15:24:44.557 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:44.533+0000 1694f700 -1 osd.5 165 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:44.614 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:44.542+0000 36a79700 -1 osd.2 165 build_incremental_map_msg missing incremental map 165 2019-12-09T15:24:44.635 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:44.612+0000 1694f700 -1 osd.2 165 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:44.682 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:24:44.529+0000 1694f700 -1 osd.4 165 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.112127+0000 front 2019-12-09T15:23:56.096301+0000 (oldest deadline 2019-12-09T15:24:41.125864+0000) 2019-12-09T15:24:45.374 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:45.350+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:45.395 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:45.378+0000 1694f700 -1 osd.0 166 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:45.429 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:45.417+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:45.531 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:45.506+0000 1694f700 -1 osd.3 166 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:45.542 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:45.537+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:45.660 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:45.656+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:46.070 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:24:46.014+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.112127+0000 front 2019-12-09T15:23:56.096301+0000 (oldest deadline 2019-12-09T15:24:41.125864+0000) 2019-12-09T15:24:46.424 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:46.421+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:46.425 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:46.422+0000 1694f700 -1 osd.0 167 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:46.479 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:46.457+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:46.571 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:46.567+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:46.582 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:46.577+0000 1694f700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:46.667 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:46.662+0000 1694f700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:46.975 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:24:46.970+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.112127+0000 front 2019-12-09T15:23:56.096301+0000 (oldest deadline 2019-12-09T15:24:41.125864+0000) 2019-12-09T15:24:47.439 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:47.432+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:47.440 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:47.436+0000 1694f700 -1 osd.0 167 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:47.492 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:47.487+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:47.621 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:47.617+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:47.623 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:47.618+0000 1694f700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:47.687 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:47.684+0000 1694f700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:47.952 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:24:47.947+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.112127+0000 front 2019-12-09T15:23:56.096301+0000 (oldest deadline 2019-12-09T15:24:41.125864+0000) 2019-12-09T15:24:48.494 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:48.490+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:48.503 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:48.499+0000 1694f700 -1 osd.0 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:48.507 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:48.502+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:48.581 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:48.576+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:48.614 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:48.610+0000 1694f700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:48.645 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:48.641+0000 1694f700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:48.957 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:24:48.951+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.112127+0000 front 2019-12-09T15:23:56.096301+0000 (oldest deadline 2019-12-09T15:24:41.125864+0000) 2019-12-09T15:24:49.487 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:49.480+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:49.487 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:49.484+0000 1694f700 -1 osd.0 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:49.512 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:49.508+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:49.599 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:49.595+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:49.605 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:49.602+0000 1694f700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:49.639 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:49.634+0000 1694f700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:49.967 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:24:49.963+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.112127+0000 front 2019-12-09T15:23:56.096301+0000 (oldest deadline 2019-12-09T15:24:41.125864+0000) 2019-12-09T15:24:50.521 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:50.515+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:50.522 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:50.518+0000 1694f700 -1 osd.0 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:50.550 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:50.544+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:50.573 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:50.567+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:50.626 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:50.620+0000 1694f700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:50.635 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:50.631+0000 1694f700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:50.942 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:24:50.936+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.112127+0000 front 2019-12-09T15:23:56.096301+0000 (oldest deadline 2019-12-09T15:24:41.125864+0000) 2019-12-09T15:24:51.495 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:51.489+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:51.496 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:51.491+0000 1694f700 -1 osd.0 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:51.553 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:51.548+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:51.585 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:51.577+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:51.593 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:51.589+0000 1694f700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:51.612 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:51.606+0000 1694f700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:51.925 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:24:51.918+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.112127+0000 front 2019-12-09T15:23:56.096301+0000 (oldest deadline 2019-12-09T15:24:41.125864+0000) 2019-12-09T15:24:52.479 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:52.471+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.555597+0000 front 2019-12-09T15:23:56.694157+0000 (oldest deadline 2019-12-09T15:24:41.198361+0000) 2019-12-09T15:24:52.479 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:52.476+0000 1694f700 -1 osd.0 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:52.531 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:24:52.528+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.074490+0000 front 2019-12-09T15:23:56.036302+0000 (oldest deadline 2019-12-09T15:24:40.716879+0000) 2019-12-09T15:24:52.582 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:24:52.577+0000 1694f700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:56.678134+0000 front 2019-12-09T15:23:59.202022+0000 (oldest deadline 2019-12-09T15:24:38.622846+0000) 2019-12-09T15:24:52.615 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:24:52.608+0000 1694f700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:59.201912+0000 front 2019-12-09T15:23:59.203764+0000 (oldest deadline 2019-12-09T15:24:41.145654+0000) 2019-12-09T15:24:52.621 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:24:52.615+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:23:57.466752+0000 front 2019-12-09T15:23:59.201756+0000 (oldest deadline 2019-12-09T15:24:39.724666+0000) 2019-12-09T15:24:53.842 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:53.811+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:54.826 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:24:54.821+0000 1694f700 -1 osd.0 169 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:112228 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5465496896 gen 1] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T15:24:55.263 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T15:24:55.215+0000 20e25700 -1 RGWWatcher::handle_error cookie 5465496896 err (107) Transport endpoint is not connected 2019-12-09T15:24:58.732 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:24:58.705+0000 1a89c700 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 1 pg recovery_unfound (PG_DAMAGED) 2019-12-09T15:25:24.096 INFO:teuthology.orchestra.run.smithi035.stderr:testCopyFromHeader404s (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:25:25.993 INFO:teuthology.orchestra.run.smithi035.stderr:testCopyNoDestinationHeader (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:25:31.567 INFO:teuthology.orchestra.run.smithi035.stderr:testDelete (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:25:32.901 INFO:teuthology.orchestra.run.smithi035.stderr:testDeleteOfFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:25:37.471 INFO:teuthology.orchestra.run.smithi035.stderr:testDeleteThen404s (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:25:38.798 INFO:teuthology.orchestra.run.smithi035.stderr:testEtagResponse (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:25:38.917 INFO:teuthology.orchestra.run.smithi035.stderr:testEtagWayoff (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:25:40.509 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:25:40.505+0000 35276700 -1 osd.2 169 build_incremental_map_msg missing incremental map 165 2019-12-09T15:25:58.753 INFO:teuthology.orchestra.run.smithi035.stderr:testFileCreate (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:26:01.067 INFO:teuthology.orchestra.run.smithi035.stderr:testGetContentType (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:26:01.630 INFO:teuthology.orchestra.run.smithi035.stderr:testGetOnFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:26:03.517 INFO:teuthology.orchestra.run.smithi035.stderr:testHead (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:26:07.286 INFO:teuthology.orchestra.run.smithi035.stderr:testHeadOnFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:26:39.467 INFO:teuthology.orchestra.run.smithi035.stderr:testMetadataOnPost (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:27:03.431 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:03.425+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:04.435 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:04.430+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:04.456 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:04.451+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:05.454 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:05.449+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:05.488 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:05.483+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:05.748 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:05.744+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:05.862 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:05.858+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:05.914 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:05.908+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:06.430 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:06.425+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:06.451 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:06.445+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:06.750 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:06.746+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:06.844 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:06.840+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:06.908 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:06.903+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:07.446 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:07.441+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:07.466 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:07.461+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:07.662 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:07.658+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:07.758 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:07.753+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:07.828 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:07.824+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:07.916 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:07.911+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:08.489 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:08.484+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:08.498 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:08.493+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:08.679 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:08.674+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:08.785 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:08.782+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:08.864 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:08.860+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:08.911 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:08.905+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:09.494 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:09.489+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:09.538 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:09.534+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:09.676 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:09.667+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:09.833 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:09.825+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:09.834 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:09.825+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:09.956 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:09.950+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:10.473 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:10.468+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:10.514 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:10.510+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:10.658 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:10.654+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:10.813 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:10.809+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:10.853 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:10.848+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:11.004 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:10.999+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:11.429 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:11.423+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:11.513 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:11.508+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:11.665 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:11.661+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:11.779 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:11.776+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:11.882 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:11.876+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:12.020 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:12.015+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:12.405 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:12.397+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:12.480 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:12.475+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:12.642 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:12.638+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:12.784 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:12.779+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:12.846 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:12.843+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:13.048 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:13.042+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:13.375 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:13.371+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:13.446 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:13.441+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:13.644 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:13.640+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:13.813 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:13.809+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:13.860 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:13.857+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:14.054 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:14.049+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:14.351 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:14.346+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:14.491 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:14.487+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:14.655 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:14.651+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:14.818 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:14.814+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:14.830 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:14.826+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:15.086 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:15.080+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:15.342 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:15.337+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:15.498 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:15.492+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:15.670 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:15.665+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:15.724 INFO:teuthology.orchestra.run.smithi035.stderr:testMetadataOnPut (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:27:15.853 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:15.848+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:15.871 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:15.868+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:16.050 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:16.045+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:16.381 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:16.375+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:16.480 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:16.475+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:16.630 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:16.626+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:16.824 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:16.819+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:16.854 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:16.849+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:17.066 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:17.060+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:17.391 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:17.385+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:17.452 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:17.442+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:17.671 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:17.667+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:17.853 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:17.843+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:17.857 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:17.850+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:18.033 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:18.024+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:18.387 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:18.383+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:18.413 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:18.406+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:18.724 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:18.720+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:18.873 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:18.868+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:18.882 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:18.870+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:19.021 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:19.015+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:19.371 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:19.365+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:19.406 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:19.401+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:19.697 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:19.692+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:19.873 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:19.865+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:19.876 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:19.872+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:20.036 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:20.030+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:20.385 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:20.378+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:20.393 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:20.384+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:20.667 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:20.661+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:20.891 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:20.888+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:20.921 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:20.910+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:21.072 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:21.067+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:21.402 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:21.397+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:21.414 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:21.410+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:21.721 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:21.712+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:21.911 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:21.908+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:21.928 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:21.914+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:22.111 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:22.104+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:22.379 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:22.373+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:22.452 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:22.445+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:22.763 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:22.758+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:22.888 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:22.884+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:22.913 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:22.909+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:23.113 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:23.108+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:23.420 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:23.415+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:23.447 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:23.442+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:23.805 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:23.801+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:23.877 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:23.873+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:23.966 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:23.961+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:24.154 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:24.148+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:24.430 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:24.425+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:24.434 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:24.429+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:24.759 INFO:teuthology.orchestra.run.smithi035.stderr:testNameLimit (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:27:24.824 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:24.820+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:24.853 INFO:teuthology.orchestra.run.smithi035.stderr:testNoContentLengthForPut (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:27:24.887 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:24.883+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:25.019 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:25.012+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:25.161 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:25.150+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:25.398 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:25.394+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:25.469 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:25.464+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:25.871 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:25.852+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:25.927 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:25.923+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:25.954 INFO:teuthology.orchestra.run.smithi035.stderr:testPostOnFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:27:26.071 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:26.067+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:26.193 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:26.187+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:26.357 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:26.351+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:26.427 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:26.422+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:26.908 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:26.902+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:26.915 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:26.910+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:27.109 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:27.098+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:27.215 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:27.210+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:27.332 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:27.326+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:27.452 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:27.447+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:27.889 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:27.885+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:27.952 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:27.947+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:28.080 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:28.075+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:28.238 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:28.231+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:28.328 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:28.323+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:28.471 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:28.466+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:28.874 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:28.870+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:28.943 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:28.938+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:29.044 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:29.039+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:29.226 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:29.221+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:29.292 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:29.287+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:29.465 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:29.453+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:29.916 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:29.912+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:29.982 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:29.977+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:30.040 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:30.035+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:30.216 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:30.211+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:30.310 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:30.304+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:30.473 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:30.464+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:30.941 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:30.933+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:31.029 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:31.025+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:31.048 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:31.040+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:31.194 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:31.187+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:31.355 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:31.348+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:31.433 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:31.427+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:31.945 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:31.937+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:32.033 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:32.025+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:32.085 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:32.079+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:32.229 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:32.222+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:32.315 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:32.309+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:32.444 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:32.439+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:27:32.983 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:27:32.976+0000 1694f700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.273534+0000 front 2019-12-09T15:26:23.355359+0000 (oldest deadline 2019-12-09T15:27:07.305528+0000) 2019-12-09T15:27:33.032 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:27:33.026+0000 1694f700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:19.210289+0000 front 2019-12-09T15:26:20.643525+0000 (oldest deadline 2019-12-09T15:27:05.059885+0000) 2019-12-09T15:27:33.081 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:27:33.076+0000 1694f700 -1 osd.3 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.195886+0000 front 2019-12-09T15:26:22.955633+0000 (oldest deadline 2019-12-09T15:27:05.593346+0000) 2019-12-09T15:27:33.288 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:27:33.278+0000 1694f700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:21.822485+0000 front 2019-12-09T15:26:21.860950+0000 (oldest deadline 2019-12-09T15:27:05.034888+0000) 2019-12-09T15:27:33.352 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:27:33.346+0000 1694f700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.524391+0000 front 2019-12-09T15:26:21.536711+0000 (oldest deadline 2019-12-09T15:27:03.418666+0000) 2019-12-09T15:27:33.447 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:27:33.440+0000 1694f700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:26:23.699479+0000 front 2019-12-09T15:26:23.374888+0000 (oldest deadline 2019-12-09T15:27:04.316667+0000) 2019-12-09T15:28:17.785 INFO:teuthology.orchestra.run.smithi035.stderr:testRangedGets (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:28:29.275 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:16:05.540 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:28:29.287 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:16:05.552 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:28:29.305 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:16:05.527 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:28:29.313 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:16:05.529 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:28:29.313 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:16:05.531 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:28:29.313 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:16:05.530 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:28:29.605 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:16:05.870 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:28:29.759 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:16:05.980 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:28:29.774 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:16:05.997 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:28:30.114 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:28:30.101+0000 35276700 -1 osd.6 178 build_incremental_map_msg missing incremental map 178 2019-12-09T15:28:30.839 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:28:30.634+0000 34274700 -1 osd.6 178 build_incremental_map_msg missing incremental map 178 2019-12-09T15:28:30.839 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:28:30.636+0000 3b282700 -1 osd.6 178 build_incremental_map_msg missing incremental map 178 2019-12-09T15:28:31.023 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:28:30.826+0000 35a77700 -1 osd.6 178 build_incremental_map_msg missing incremental map 178 2019-12-09T15:28:31.069 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:30.981+0000 1694f700 -1 osd.0 178 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:32.058 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:32.015+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:32.154 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:32.147+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:32.785 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:32.780+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:33.105 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:33.100+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:33.111 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:33.104+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:33.754 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:33.747+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:33.886 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:33.881+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:34.107 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:34.101+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:34.107 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:34.104+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:34.449 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:34.444+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:34.773 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:28:34.761+0000 3a280700 -1 osd.6 179 build_incremental_map_msg missing incremental map 178 2019-12-09T15:28:34.791 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:28:34.765+0000 3a280700 -1 osd.6 179 build_incremental_map_msg missing incremental map 178 2019-12-09T15:28:34.791 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:34.787+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:34.799 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:34.793+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:34.871 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:34.861+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:35.156 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:35.152+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:35.157 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:35.153+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:35.480 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:35.475+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:35.543 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:35.539+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:35.763 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:35.756+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:35.806 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:35.800+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:35.870 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:35.857+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:36.199 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:36.194+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:36.201 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:36.198+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:36.459 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:36.456+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:36.561 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:36.556+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:36.763 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:36.757+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:36.766 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:36.760+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:36.901 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:36.893+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:37.206 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:37.201+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:37.208 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:37.203+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:37.425 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:37.418+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:37.546 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:37.541+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:37.739 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:37.733+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:37.775 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:37.768+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:37.897 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:37.890+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:38.204 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:38.198+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:38.205 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:38.201+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:38.436 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:38.431+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:38.598 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:38.592+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:38.773 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:38.762+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:38.809 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:38.802+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:38.889 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:38.882+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:39.211 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:39.206+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:39.212 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:39.208+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:39.412 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:39.407+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:39.637 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:39.634+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:39.793 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:39.786+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:39.796 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:39.789+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:39.916 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:39.910+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:40.201 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:40.195+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:40.204 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:40.201+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:40.439 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:40.432+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:40.609 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:40.604+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:40.752 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:40.746+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:40.847 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:40.840+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:40.886 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:40.875+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:41.218 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:41.210+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:41.219 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:41.215+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:41.483 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:41.477+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:41.661 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:41.655+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:41.744 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:41.738+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:41.841 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:41.835+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:41.912 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:41.905+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:42.180 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:42.174+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:42.188 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:42.177+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:42.441 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:42.436+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:42.704 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:42.698+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:42.785 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:42.779+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:42.865 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:42.858+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:42.897 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:42.890+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:43.235 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:43.228+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:43.236 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:43.232+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:43.437 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:43.431+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:43.722 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:43.718+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:43.778 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:43.771+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:43.882 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:43.876+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:43.883 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:43.878+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:44.222 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:44.211+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:44.225 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:44.214+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:44.429 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:44.425+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:44.716 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:44.711+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:44.769 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:44.758+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:44.894 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:44.888+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:44.900 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:44.894+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:45.203 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:45.198+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:45.204 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:45.200+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:45.432 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:45.427+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:45.684 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:45.679+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:45.729 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:45.721+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:45.922 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:45.915+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:45.922 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:45.916+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:46.231 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:46.227+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:46.233 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:46.230+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:46.394 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:46.390+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:46.682 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:46.677+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:46.703 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:46.696+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:46.949 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:46.942+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:46.962 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:46.954+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:47.250 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:47.245+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:47.253 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:47.249+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:47.391 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:47.387+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:47.675 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:47.671+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:47.687 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:47.681+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:47.922 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:47.915+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:47.937 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:47.931+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:48.244 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:48.239+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:48.245 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:48.241+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:48.353 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:48.349+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:48.637 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:48.632+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:48.736 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:48.729+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:48.930 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:48.924+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:48.960 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:48.953+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:49.236 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:49.229+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:49.237 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:49.232+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:49.365 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:49.359+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:49.683 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:49.678+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:49.795 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:49.788+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:49.902 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:49.897+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:50.000 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:49.994+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:50.285 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:50.278+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:50.286 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:50.282+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:50.356 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:50.349+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:50.660 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:50.655+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:50.766 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:50.760+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:50.879 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:50.872+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:50.970 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:50.963+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:51.262 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:51.259+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:51.264 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:51.261+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:51.325 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:51.321+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:51.692 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:51.686+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:51.731 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:51.725+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:51.899 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:51.893+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:51.945 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:51.935+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:52.230 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:52.224+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:52.240 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:52.230+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:52.321 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:52.316+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:52.653 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:52.648+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:52.771 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:52.766+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:52.929 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:28:52.923+0000 1694f700 -1 osd.5 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:48.530447+0000 front 2019-12-09T15:27:48.403647+0000 (oldest deadline 2019-12-09T15:28:33.681944+0000) 2019-12-09T15:28:52.938 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:28:52.933+0000 1694f700 -1 osd.7 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:50.153268+0000 front 2019-12-09T15:27:50.233266+0000 (oldest deadline 2019-12-09T15:28:33.988136+0000) 2019-12-09T15:28:53.285 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:53.280+0000 1694f700 -1 osd.0 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.278908+0000 front 2019-12-09T15:27:46.310906+0000 (oldest deadline 2019-12-09T15:28:31.304694+0000) 2019-12-09T15:28:53.286 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:53.283+0000 1694f700 -1 osd.0 179 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:53.333 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:28:53.328+0000 1694f700 -1 osd.2 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.194489+0000 front 2019-12-09T15:27:48.489984+0000 (oldest deadline 2019-12-09T15:28:34.181484+0000) 2019-12-09T15:28:53.642 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:28:53.638+0000 1694f700 -1 osd.3 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:49.529971+0000 front 2019-12-09T15:27:50.397960+0000 (oldest deadline 2019-12-09T15:28:35.424476+0000) 2019-12-09T15:28:53.820 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:28:53.815+0000 1694f700 -1 osd.4 179 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:27:46.136493+0000 front 2019-12-09T15:27:46.329613+0000 (oldest deadline 2019-12-09T15:28:31.887823+0000) 2019-12-09T15:28:54.659 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:54.649+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:55.832 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:28:55.708+0000 1694f700 -1 osd.0 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113007 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418077328 gen 3] snapc 0=[] ondisk+write+known_if_redirected e175) 2019-12-09T15:28:56.438 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T15:28:56.430+0000 20e25700 -1 RGWWatcher::handle_error cookie 5418077328 err (107) Transport endpoint is not connected 2019-12-09T15:29:56.624 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:29:56.433+0000 1694f700 -1 osd.1 180 get_health_metrics reporting 1 slow ops, oldest is osd_repop(osd.0.0:29214 5.5 e175/171 5:a93a5511:::notify.2:head v 175'178) 2019-12-09T15:30:04.202 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:17:40.466 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:04.221 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:17:40.443 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:04.221 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:17:40.486 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:04.227 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:17:40.450 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:04.228 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:17:40.451 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:04.238 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:17:40.460 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:04.346 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:17:40.568 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:04.359 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:17:40.580 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:04.508 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:17:40.772 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:58.239 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:30:58.234+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:30:59.213 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:30:59.208+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:30:59.430 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:30:59.425+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:30:59.641 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:30:59.636+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:00.241 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:00.236+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:00.473 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:00.469+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:00.602 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:00.599+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:01.293 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:01.289+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:01.368 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:01.361+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:01.453 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:01.448+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:01.606 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:01.602+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:02.215 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:02.209+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:02.326 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:02.320+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:02.355 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:02.350+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:02.445 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:02.440+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:02.643 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:02.638+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:03.221 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:03.214+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:03.309 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:03.304+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:03.319 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:03.316+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:03.491 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:03.485+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:03.675 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:03.670+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:03.840 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:03.835+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:04.271 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:04.266+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:04.274 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:04.268+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:04.355 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:04.350+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:04.523 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:04.518+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:04.644 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:04.639+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:04.872 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:04.860+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:05.271 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:05.264+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:05.317 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:05.312+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:05.350 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:05.345+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:05.531 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:05.526+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:05.623 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:05.618+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:05.852 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:05.847+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:06.283 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:06.277+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:06.370 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:06.366+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:06.381 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:06.377+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:06.567 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:06.562+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:06.579 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:06.575+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:06.857 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:06.852+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:07.305 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:07.300+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:07.349 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:07.346+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:07.408 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:07.403+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:07.553 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:07.549+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:07.563 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:07.558+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:07.901 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:07.897+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:08.326 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:08.320+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:08.348 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:08.344+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:08.400 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:08.395+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:08.529 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:08.524+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:08.614 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:08.610+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:08.892 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:08.888+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:09.302 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:09.296+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:09.367 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:09.361+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:09.371 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:09.367+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:09.579 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:09.571+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:09.661 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:09.657+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:09.877 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:09.873+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:10.323 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:10.316+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:10.381 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:10.376+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:10.403 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:10.398+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:10.629 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:10.624+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:10.706 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:10.701+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:10.877 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:10.873+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:11.361 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:11.355+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:11.383 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:11.379+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:11.430 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:11.425+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:11.655 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:11.650+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:11.697 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:11.693+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:11.853 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:11.849+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:12.343 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:12.338+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:12.354 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:12.347+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:12.415 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:12.409+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:12.650 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:12.645+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:12.737 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:12.712+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:12.870 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:12.866+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:13.331 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:13.326+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:13.347 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:13.342+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:13.380 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:13.376+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:13.673 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:13.668+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:13.720 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:13.715+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:13.888 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:13.884+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:14.310 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:14.305+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:14.351 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:14.346+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:14.373 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:14.368+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:14.707 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:14.704+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:14.743 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:14.739+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:14.922 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:14.917+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:15.343 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:15.337+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:15.391 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:15.386+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:15.408 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:15.402+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:15.698 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:15.693+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:15.707 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:15.700+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:15.885 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:15.881+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:16.325 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:16.320+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:16.351 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:16.347+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:16.395 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:16.389+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:16.685 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:16.681+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:16.714 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:16.710+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:16.845 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:16.840+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:17.060 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:17.055+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:17.309 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:17.304+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:17.363 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:17.358+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:17.416 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:17.411+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:17.679 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:17.675+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:17.694 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:17.690+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:17.892 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:17.888+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:18.051 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:18.046+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:18.313 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:18.310+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:18.337 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:18.332+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:18.428 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:18.422+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:18.658 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:18.646+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:18.687 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:18.682+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:18.912 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:18.908+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:19.088 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:19.083+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:19.351 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:19.346+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:19.371 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:19.365+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:19.424 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:19.418+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:19.615 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:19.610+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:19.661 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:19.657+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:19.949 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:19.945+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:20.130 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:20.123+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:20.326 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:20.320+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:20.351 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:20.347+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:20.418 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:20.412+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:20.635 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:20.631+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:20.672 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:20.668+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:20.950 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:20.946+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:21.151 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:21.146+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:21.364 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:21.359+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:21.377 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:21.373+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:21.415 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:21.409+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:21.642 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:21.638+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:21.674 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:21.670+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:21.950 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:21.946+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:22.182 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:22.176+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:22.365 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:22.360+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:22.412 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:22.406+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:22.440 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:22.435+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:22.687 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:22.682+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:22.693 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:22.688+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:22.957 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:22.952+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:23.223 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:23.218+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:23.384 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:23.380+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:23.385 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:23.381+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:23.429 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:23.425+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:23.666 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:23.659+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:23.714 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:23.709+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:23.960 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:23.954+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:24.236 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:24.233+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:24.342 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:24.338+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:24.360 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:24.356+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:24.388 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:24.384+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:24.669 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:24.665+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:24.725 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:24.713+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:24.940 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:24.933+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:25.229 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:25.225+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:25.351 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:25.347+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:25.354 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:25.350+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:25.428 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:25.424+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:25.624 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:25.620+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:25.766 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:25.759+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:25.933 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:25.928+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:26.187 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:26.182+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:26.324 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:26.319+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:26.388 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:26.384+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:26.435 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:26.431+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:26.620 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:26.616+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:26.788 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:26.783+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:26.970 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:26.965+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:27.202 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:27.198+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:27.367 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:27.363+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:27.370 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:27.366+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:27.472 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:27.467+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:27.657 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:27.653+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:27.805 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:27.800+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:27.997 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:27.985+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:28.193 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:28.185+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:28.333 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:28.329+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:28.370 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:28.366+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:28.447 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:28.443+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:28.715 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:28.710+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:28.797 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:28.792+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:29.016 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:29.011+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:29.155 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:29.150+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:29.308 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:29.302+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:29.398 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:29.394+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:29.471 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:29.466+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:29.709 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:29.705+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:29.817 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:29.813+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:30.067 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:30.062+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:30.198 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:30.193+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:30.328 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:30.323+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:30.431 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:30.427+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:30.457 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:30.452+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:30.696 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:30.692+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:30.809 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:30.805+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:31.086 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:31.080+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:31.167 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:31.161+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:31.329 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:31.323+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:31.433 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:31.429+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:31.461 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:31.457+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:31.698 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:31.693+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:31.780 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:31.774+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:32.129 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:32.125+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:32.185 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:32.180+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:32.289 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:32.284+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:32.469 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:32.462+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:32.506 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:32.502+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:32.698 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:32.692+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:32.828 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:32.822+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:33.128 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:33.123+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:33.234 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:33.229+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:33.291 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:33.285+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:33.441 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:33.436+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:33.547 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:33.544+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:33.680 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:33.674+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:33.811 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:33.806+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:34.159 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:34.154+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:34.258 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:34.253+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:34.280 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:34.276+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:34.414 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:34.409+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:34.560 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:34.555+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:34.583 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:31:34.574+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 38sec e186 v186) 2019-12-09T15:31:34.713 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:34.708+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:34.853 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:34.849+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:35.167 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:35.162+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:35.239 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:35.234+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:35.252 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:35.248+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:35.370 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:35.367+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:35.584 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:35.579+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:35.754 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:35.750+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:35.874 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:35.869+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:36.182 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:36.178+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:36.238 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:36.233+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:36.294 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:36.290+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:36.421 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:36.417+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:36.551 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:36.547+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:36.742 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:36.738+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:36.912 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:36.907+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:37.234 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:37.230+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:37.268 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:37.262+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:37.300 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:37.295+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:37.396 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:37.392+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:37.543 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:37.539+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:37.716 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:37.711+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:37.966 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:37.961+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:38.239 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:38.234+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:38.245 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:38.241+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:38.305 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:38.301+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:38.369 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:38.363+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:38.594 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:38.588+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:38.686 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:38.682+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:31:38.974 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:31:38.969+0000 1694f700 -1 osd.3 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.859337+0000 front 2019-12-09T15:30:21.200257+0000 (oldest deadline 2019-12-09T15:30:59.476403+0000) 2019-12-09T15:31:39.207 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:31:39.203+0000 1694f700 -1 osd.1 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.191815+0000 front 2019-12-09T15:30:21.102198+0000 (oldest deadline 2019-12-09T15:30:58.061427+0000) 2019-12-09T15:31:39.257 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:31:39.253+0000 1694f700 -1 osd.0 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:20.552725+0000 front 2019-12-09T15:30:20.617476+0000 (oldest deadline 2019-12-09T15:31:03.070843+0000) 2019-12-09T15:31:39.354 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:31:39.349+0000 1694f700 -1 osd.7 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:16.833797+0000 front 2019-12-09T15:30:16.445303+0000 (oldest deadline 2019-12-09T15:31:16.423706+0000) 2019-12-09T15:31:39.405 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:31:39.400+0000 1694f700 -1 osd.4 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:21.096492+0000 front 2019-12-09T15:30:18.060459+0000 (oldest deadline 2019-12-09T15:31:02.201430+0000) 2019-12-09T15:31:39.584 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:31:39.579+0000 1694f700 -1 osd.5 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:15.608139+0000 front 2019-12-09T15:30:16.832934+0000 (oldest deadline 2019-12-09T15:31:00.924090+0000) 2019-12-09T15:31:39.671 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:31:39.664+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 38sec e186 v186) 2019-12-09T15:31:39.727 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:31:39.723+0000 1694f700 -1 osd.2 186 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:30:18.638193+0000 front 2019-12-09T15:30:18.648943+0000 (oldest deadline 2019-12-09T15:30:58.728677+0000) 2019-12-09T15:32:34.167 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:20:10.431 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:32:34.173 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:20:10.438 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:32:34.185 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:20:10.407 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:32:34.199 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:20:10.409 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:32:34.200 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:20:10.410 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:32:34.200 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:20:10.414 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:32:34.330 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:20:10.594 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:32:34.350 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:20:10.573 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:32:34.363 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:20:10.585 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:32:35.809 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:35.806+0000 1694f700 -1 osd.0 189 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:36.764 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:36.761+0000 1694f700 -1 osd.0 189 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:37.103 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:37.078+0000 37a7b700 -1 osd.5 190 build_incremental_map_msg missing incremental map 190 2019-12-09T15:32:37.309 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:37.234+0000 3a280700 -1 osd.5 190 build_incremental_map_msg missing incremental map 190 2019-12-09T15:32:37.492 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:37.390+0000 3aa81700 -1 osd.5 190 build_incremental_map_msg missing incremental map 190 2019-12-09T15:32:37.582 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:37.578+0000 11caf700 -1 osd.5 190 build_incremental_map_msg missing incremental map 190 2019-12-09T15:32:37.621 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:37.616+0000 1694f700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:37.740 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:37.737+0000 1694f700 -1 osd.0 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:38.603 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:38.598+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:38.638 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:38.633+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:38.749 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:38.744+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:39.345 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:39.341+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:39.560 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:39.556+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:39.695 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:39.685+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:39.765 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:39.757+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:39.771 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:39.765+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:40.381 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:40.377+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:40.534 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:40.530+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:40.542 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:40.536+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:40.749 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:40.746+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:40.793 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:40.789+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:40.793 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:40.790+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:41.352 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:41.348+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:41.537 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:41.527+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:41.648 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:41.608+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:41.742 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:41.738+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:41.815 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:41.810+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:41.816 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:41.812+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:41.991 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:41.984+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:42.348 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:42.342+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:42.513 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:42.510+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:42.589 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:42.586+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:42.768 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:42.764+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:42.768 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:42.765+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:42.776 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:42.772+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:43.009 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:43.005+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:43.313 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:43.307+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:43.495 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:43.490+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:43.576 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:43.569+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:43.728 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:43.724+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:43.730 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:43.727+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:43.752 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:43.747+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:43.994 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:43.988+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:44.332 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:44.327+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:44.494 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:44.489+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:44.541 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:44.534+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:44.768 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:44.764+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:44.775 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:44.770+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:44.778 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:44.775+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:44.991 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:44.987+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:45.313 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:45.309+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:45.484 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:45.479+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:45.571 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:45.566+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:45.800 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:45.790+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:45.800 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:45.794+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:45.820 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:45.815+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:46.044 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:46.039+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:46.329 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:46.323+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:46.471 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:46.466+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:46.530 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:46.525+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:46.822 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:46.817+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:46.825 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:46.821+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:46.825 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:46.822+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:47.067 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:47.061+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:47.306 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:47.301+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:47.446 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:47.440+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:47.517 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:47.511+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:47.785 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:47.778+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:47.857 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:47.853+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:47.893 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:47.854+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:48.120 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:48.115+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:48.310 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:48.304+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:48.407 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:48.403+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:48.513 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:48.507+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:48.776 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:48.771+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:48.900 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:48.894+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:48.901 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:48.897+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:49.172 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:49.167+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:49.356 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:49.350+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:49.395 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:49.389+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:49.502 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:49.496+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:49.796 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:49.791+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:49.857 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:49.853+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:49.858 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:49.855+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:50.217 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:50.208+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:50.335 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:50.330+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:50.416 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:50.410+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:50.478 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:50.472+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:50.806 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:50.801+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:50.863 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:50.859+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:50.869 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:50.863+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:51.241 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:51.236+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:51.310 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:51.306+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:51.380 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:51.375+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:51.498 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:51.492+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:51.820 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:51.815+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:51.821 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:51.815+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:51.824 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:51.820+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:52.246 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:52.241+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:52.337 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:52.332+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:52.384 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:52.379+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:52.457 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:52.452+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:52.780 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:52.774+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:52.848 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:52.842+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:52.848 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:52.845+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:53.285 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:53.280+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:53.314 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:53.309+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:53.383 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:53.377+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:53.463 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:53.457+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:53.809 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:53.803+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:53.838 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:53.833+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:53.839 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:53.835+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:54.265 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:54.260+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:54.309 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:54.303+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:54.397 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:54.391+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:54.513 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:54.507+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:54.807 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:54.802+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:54.886 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:54.879+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:54.887 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:54.883+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:55.284 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:55.279+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:55.305 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:55.299+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:55.376 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:55.369+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:55.513 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:55.506+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:55.807 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:55.802+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:55.932 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:55.927+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:55.933 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:55.929+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:56.298 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:56.294+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:56.323 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:56.319+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:56.381 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:56.375+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:56.541 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:56.534+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:56.835 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:56.828+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:56.910 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:56.904+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:56.911 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:56.907+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:57.276 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:57.271+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:57.329 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:57.324+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:57.385 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:57.379+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:57.532 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:57.525+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:57.802 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:57.795+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:57.941 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:57.935+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:57.942 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:57.938+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:58.306 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:58.301+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:58.322 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:58.318+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:58.424 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:58.416+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:58.506 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:58.499+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:58.824 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:58.818+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:58.927 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:58.915+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:58.928 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:58.922+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:32:59.291 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:32:59.286+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:32:59.354 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:32:59.348+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:32:59.478 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:32:59.471+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:32:59.488 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:32:59.482+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:32:59.794 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:32:59.788+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:32:59.960 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:59.954+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:32:59.961 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:32:59.957+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:00.310 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:33:00.305+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:33:00.377 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:33:00.372+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:33:00.437 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:33:00.430+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:33:00.517 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:33:00.511+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:33:00.802 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:33:00.796+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:33:00.988 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:00.983+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:33:00.989 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:00.986+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:01.288 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:33:01.285+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:33:01.347 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:33:01.343+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:33:01.391 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:33:01.386+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:33:01.500 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:33:01.494+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:33:01.768 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:33:01.762+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:33:01.945 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:01.939+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:33:01.946 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:01.942+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:02.341 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:33:02.336+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:33:02.348 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:33:02.343+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:33:02.357 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:33:02.352+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:33:02.503 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:33:02.496+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:33:02.812 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:33:02.806+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:33:02.947 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:02.941+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:33:02.950 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:02.947+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:03.326 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:33:03.322+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:33:03.378 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:33:03.371+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:33:03.397 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:33:03.385+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:33:03.471 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:33:03.463+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:33:03.862 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:33:03.855+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:33:03.970 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:03.966+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:33:03.981 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:03.977+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:04.351 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:33:04.344+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:33:04.367 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:33:04.361+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:33:04.425 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:33:04.420+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:33:04.433 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:33:04.427+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:33:04.886 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:33:04.878+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:33:05.029 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:05.022+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:33:05.030 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:05.026+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:05.361 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:33:05.354+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:33:05.380 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:33:05.375+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:33:05.405 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:33:05.392+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:33:05.469 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:33:05.464+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:33:05.897 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:33:05.890+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:33:06.070 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:06.063+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:33:06.071 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:06.067+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:06.338 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:33:06.331+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:33:06.401 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:33:06.395+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:33:06.419 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:33:06.414+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:33:06.506 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:33:06.502+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:33:06.943 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:33:06.936+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:33:07.100 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:07.092+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:33:07.101 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:07.096+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:07.293 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:33:07.288+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:33:07.403 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:33:07.396+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:33:07.428 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:33:07.421+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:33:07.502 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:33:07.495+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:33:07.935 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:33:07.927+0000 1694f700 -1 osd.6 192 get_health_metrics reporting 43 slow ops, oldest is osd_op(client.4355.0:111662 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e190) 2019-12-09T15:33:07.965 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:33:07.959+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:33:08.104 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:08.095+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:33:08.105 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:08.099+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:08.298 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:33:08.291+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:33:08.405 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:33:08.399+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:33:08.439 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:33:08.432+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:33:08.486 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:33:08.481+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:33:08.965 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:33:08.958+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:33:09.010 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:33:08.991+0000 1694f700 -1 osd.6 192 get_health_metrics reporting 169 slow ops, oldest is osd_op(client.4355.0:111662 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e190) 2019-12-09T15:33:09.142 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:09.135+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:33:09.143 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:09.138+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:09.307 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:33:09.299+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:33:09.422 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:33:09.415+0000 1694f700 -1 osd.7 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.808342+0000 front 2019-12-09T15:31:55.659588+0000 (oldest deadline 2019-12-09T15:32:39.885136+0000) 2019-12-09T15:33:09.430 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:33:09.423+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:33:09.536 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:33:09.530+0000 1694f700 -1 osd.3 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:56.802125+0000 front 2019-12-09T15:31:56.228363+0000 (oldest deadline 2019-12-09T15:32:41.362198+0000) 2019-12-09T15:33:09.941 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:33:09.934+0000 1694f700 -1 osd.5 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.930056+0000 front 2019-12-09T15:31:53.445372+0000 (oldest deadline 2019-12-09T15:32:37.373756+0000) 2019-12-09T15:33:10.051 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:33:10.035+0000 1694f700 -1 osd.6 192 get_health_metrics reporting 169 slow ops, oldest is osd_op(client.4355.0:111662 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e190) 2019-12-09T15:33:10.155 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:10.149+0000 1694f700 -1 osd.0 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.230161+0000 front 2019-12-09T15:31:54.254167+0000 (oldest deadline 2019-12-09T15:32:39.501209+0000) 2019-12-09T15:33:10.159 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:10.154+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:10.171 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:33:10.158+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e192 v192) 2019-12-09T15:33:10.296 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:33:10.288+0000 1694f700 -1 osd.4 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:54.885519+0000 front 2019-12-09T15:31:54.849804+0000 (oldest deadline 2019-12-09T15:32:37.730197+0000) 2019-12-09T15:33:10.481 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:33:10.476+0000 1694f700 -1 osd.2 192 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:31:55.267257+0000 front 2019-12-09T15:31:55.658290+0000 (oldest deadline 2019-12-09T15:32:38.374972+0000) 2019-12-09T15:33:11.170 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:11.167+0000 1694f700 -1 osd.0 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:11.434 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:33:11.336+0000 1694f700 -1 osd.6 193 get_health_metrics reporting 87 slow ops, oldest is osd_op(client.4355.0:112642 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e190) 2019-12-09T15:33:12.254 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:33:12.224+0000 1694f700 -1 osd.0 194 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:113473 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 363411872 gen 2] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T15:33:13.231 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T15:33:13.110+0000 20e25700 -1 RGWWatcher::handle_error cookie 363411872 err (107) Transport endpoint is not connected 2019-12-09T15:34:00.102 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:33:59.947+0000 1694f700 -1 osd.1 194 get_health_metrics reporting 2 slow ops, oldest is pg_update_log_missing(5.5 epoch 188/184 rep_tid 29222 entries 188'183 (0'0) error 5:a93a5511:::notify.2:head by client.4355.0:113473 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 182'181) 2019-12-09T15:34:04.983 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:34:04.980+0000 37a7b700 -1 osd.1 197 build_incremental_map_msg missing incremental map 197 2019-12-09T15:34:05.133 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:34:05.086+0000 34a75700 -1 osd.1 197 build_incremental_map_msg missing incremental map 197 2019-12-09T15:34:05.162 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:34:05.110+0000 36278700 -1 osd.1 197 build_incremental_map_msg missing incremental map 197 2019-12-09T15:34:05.383 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:34:05.178+0000 34a75700 -1 osd.1 197 build_incremental_map_msg missing incremental map 197 2019-12-09T15:34:05.437 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:34:05.265+0000 38a7d700 -1 osd.1 197 build_incremental_map_msg missing incremental map 197 2019-12-09T15:34:05.471 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:34:05.339+0000 38a7d700 -1 osd.1 197 build_incremental_map_msg missing incremental map 197 2019-12-09T15:34:05.519 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:34:05.384+0000 35276700 -1 osd.1 197 build_incremental_map_msg missing incremental map 197 2019-12-09T15:34:05.603 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:34:05.426+0000 38a7d700 -1 osd.1 197 build_incremental_map_msg missing incremental map 197 2019-12-09T15:34:05.736 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:34:05.426+0000 39a7f700 -1 osd.1 197 build_incremental_map_msg missing incremental map 197 2019-12-09T15:35:03.305 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:03.301+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:03.360 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:03.354+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:03.392 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:03.386+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:04.002 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:03.997+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:04.350 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:04.346+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:04.388 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:04.382+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:04.442 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:04.437+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:04.981 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:04.976+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:05.356 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:05.351+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:05.371 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:05.365+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:05.409 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:05.402+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:05.962 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:05.957+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:06.327 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:06.322+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:06.372 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:06.367+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:06.382 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:06.376+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:06.468 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:06.462+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:06.922 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:06.917+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:07.056 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:07.052+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:07.300 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:07.294+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:07.337 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:07.332+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:07.425 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:07.421+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:07.428 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:07.423+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:07.922 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:07.917+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:08.103 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:08.096+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:08.283 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:08.272+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:08.386 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:08.383+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:08.394 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:08.390+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:08.461 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:08.456+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:08.971 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:08.963+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:09.113 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:09.108+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:09.310 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:09.304+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:09.368 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:09.364+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:09.448 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:09.443+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:09.483 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:09.479+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:09.935 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:09.931+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:10.140 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:10.135+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:10.303 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:10.299+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:10.365 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:10.360+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:10.430 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:10.426+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:10.466 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:10.461+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:10.926 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:10.922+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:11.079 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:11.071+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:11.137 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:11.133+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:11.259 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:11.255+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:11.353 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:11.348+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:11.438 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:11.434+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:11.446 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:11.440+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:11.915 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:11.910+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:12.045 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:12.038+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:12.192 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:12.186+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:12.220 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:12.216+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:12.397 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:12.393+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:12.434 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:12.429+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:12.469 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:12.460+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:12.956 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:12.951+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:13.045 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:13.040+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:13.184 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:13.179+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:13.254 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:13.249+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:13.443 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:13.438+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:13.450 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:13.446+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:13.504 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:13.500+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:13.922 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:13.918+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:14.054 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:14.049+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:14.192 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:14.187+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:14.264 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:14.261+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:14.438 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:14.434+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:14.451 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:14.447+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:14.493 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:14.489+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:14.894 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:14.890+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:15.074 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:15.069+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:15.207 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:15.203+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:15.309 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:15.305+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:15.419 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:15.415+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:15.431 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:15.426+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:15.540 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:15.536+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:15.863 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:15.859+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:16.085 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:16.079+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:16.199 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:16.194+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:16.264 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:16.259+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:16.436 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:16.431+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:16.485 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:16.478+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:16.594 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:16.588+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:16.909 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:16.905+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:17.071 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:17.066+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:17.163 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:17.158+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:17.285 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:17.280+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:17.478 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:17.474+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:17.530 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:17.525+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:17.604 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:17.599+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:17.956 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:17.952+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:18.033 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:18.029+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:18.155 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:18.150+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:18.243 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:18.239+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:18.436 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:18.431+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:18.546 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:18.541+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:18.605 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:18.597+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:18.931 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:18.927+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:19.047 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:19.041+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:19.158 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:19.153+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:19.215 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:19.211+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:19.436 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:19.423+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:19.526 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:19.521+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:19.629 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:19.625+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:19.955 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:19.950+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:20.020 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:20.016+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:20.149 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:20.143+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:20.201 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:20.197+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:20.440 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:20.436+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:20.538 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:20.534+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:20.683 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:20.678+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:20.979 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:20.975+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:21.070 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:21.064+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:21.125 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:21.118+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:21.247 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:21.243+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:21.467 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:21.462+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:21.504 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:21.500+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:21.732 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:21.728+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:21.986 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:21.982+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:22.029 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:22.023+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:22.140 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:22.135+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:22.201 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:22.197+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:22.454 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:22.449+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:22.509 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:22.504+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:22.739 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:22.734+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:22.975 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:22.970+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:23.012 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:23.007+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:23.125 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:23.120+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:23.222 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:23.217+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:23.428 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:23.424+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:23.559 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:23.554+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:23.770 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:23.762+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:23.990 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:23.985+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:24.052 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:24.046+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:24.146 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:24.141+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:24.215 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:24.210+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:24.437 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:24.433+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:24.591 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:24.585+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:24.739 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:24.736+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:25.026 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:25.021+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:25.064 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:25.060+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:25.181 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:25.175+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:25.184 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:25.174+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:25.469 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:25.463+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:25.545 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:25.541+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:25.716 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:25.712+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:26.035 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:26.030+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:26.073 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:26.068+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:26.200 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:26.195+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:26.223 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:26.217+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:26.487 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:26.481+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:26.517 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:26.513+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:26.697 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:26.693+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:27.032 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:27.026+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:27.125 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:27.121+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:27.207 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:27.202+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:27.232 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:27.226+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:27.518 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:27.512+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:27.557 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:27.553+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:27.745 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:27.740+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:27.992 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:27.988+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:28.091 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:28.085+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:28.201 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:28.195+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:28.280 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:28.274+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:28.529 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:28.524+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:28.608 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:28.602+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:28.766 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:28.761+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:29.041 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:29.037+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:29.100 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:29.095+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:29.174 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:29.169+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:29.330 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:29.324+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:29.491 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:29.486+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:29.576 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:29.571+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:29.733 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:29.727+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:30.008 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:30.004+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:30.071 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:30.067+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:30.163 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:30.159+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:30.380 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:30.375+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:30.537 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:30.531+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:30.595 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:30.590+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:30.713 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:30.708+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:31.000 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:30.995+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:31.027 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:31.021+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:31.177 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:31.171+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:31.431 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:31.426+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:31.497 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:31.492+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:31.602 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:31.596+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:31.711 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:31.705+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:31.955 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:31.949+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:32.052 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:32.045+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:32.133 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:32.126+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:32.470 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:32.464+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:32.539 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:32.534+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:32.641 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:32.636+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:32.713 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:32.704+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:32.939 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:32.933+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:33.021 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:33.015+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:33.161 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:33.155+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:33.486 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:33.479+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:33.588 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:33.583+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:33.676 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:33.672+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:33.731 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:33.725+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:33.945 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:33.940+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:34.027 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:34.021+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:34.143 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:34.137+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:34.470 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:34.464+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:34.637 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:34.632+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:34.694 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:34.689+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:34.730 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:34.726+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:34.924 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:34.919+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:35.003 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:34.997+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:35.183 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:35.178+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:35.460 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:35.454+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:35.603 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:35.597+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:35.670 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:35.665+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:35.757 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:35.752+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:35.800 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:35:35.790+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 38sec e198 v198) 2019-12-09T15:35:35.894 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:35.889+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:35.989 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:35.984+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:36.194 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:36.190+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:36.444 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:36.439+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:36.563 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:36.558+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:36.634 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:36.629+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:36.739 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:36.734+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:36.914 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:36.910+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:36.980 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:36.975+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:37.163 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:37.158+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:37.425 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:37.419+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:37.551 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:37.545+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:37.628 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:37.623+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:37.740 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:37.735+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:37.916 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:37.912+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:37.966 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:37.961+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:38.142 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:38.138+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:38.465 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:38.459+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:38.589 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:38.584+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:38.657 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:38.653+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:38.731 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:38.727+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:38.890 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:38.885+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:38.987 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:38.981+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:39.112 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:39.108+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:39.422 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:39.417+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:39.628 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:39.623+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:39.667 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:39.662+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:39.726 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:39.722+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:39.860 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:39.856+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:40.030 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:40.025+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:40.084 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:40.078+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:40.441 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:40.436+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:40.636 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:40.631+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:40.659 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:40.654+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:40.763 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:40.759+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:40.804 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:35:40.799+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 38sec e198 v198) 2019-12-09T15:35:40.903 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:40.900+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:41.009 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:41.004+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:41.050 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:41.046+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:41.435 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:41.430+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:41.664 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:41.657+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:41.665 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:41.660+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:41.746 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:41.741+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:41.938 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:41.932+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:42.049 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:42.044+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:42.051 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:42.046+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:42.445 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:42.439+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:42.637 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:42.632+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:42.659 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:42.654+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:42.750 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:42.745+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:42.936 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:42.931+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:43.053 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:43.047+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:43.092 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:43.087+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:43.431 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:43.426+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:43.687 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:43.682+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:43.693 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:43.689+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:43.774 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:43.769+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:43.967 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:43.963+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:44.027 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:44.022+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:44.107 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:44.102+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:44.449 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:44.444+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:44.715 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:44.708+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:44.722 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:44.715+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:44.754 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:44.750+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:44.988 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:44.983+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:44.989 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:44.985+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:45.143 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:45.137+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:45.420 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:45.415+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:45.674 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:45.670+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:45.683 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:45.677+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:45.809 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:45.805+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:45.824 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:35:45.820+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 38sec e198 v198) 2019-12-09T15:35:46.000 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:45.995+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:46.006 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:46.001+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:46.110 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:46.104+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:46.404 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:46.398+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:46.692 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:46.688+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:46.727 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:46.721+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:46.866 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:46.861+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:46.971 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:46.966+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:47.029 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:47.025+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:47.084 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:47.078+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:47.407 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:47.403+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:47.686 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:47.681+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:47.743 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:47.737+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:47.874 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:47.869+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:47.934 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:47.930+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:48.040 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:48.034+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:48.043 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:48.038+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:48.431 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:48.425+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:48.683 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:48.679+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:48.700 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:48.695+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:48.908 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:48.904+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:48.925 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:48.920+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:49.021 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:49.017+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:49.049 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:49.044+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:49.473 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:49.468+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:49.678 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:49.674+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:49.733 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:49.727+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:49.929 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:49.924+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:49.940 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:49.936+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:50.029 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:35:50.023+0000 1694f700 -1 osd.4 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.122462+0000 front 2019-12-09T15:34:21.461420+0000 (oldest deadline 2019-12-09T15:35:03.968712+0000) 2019-12-09T15:35:50.035 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:50.030+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:35:50.465 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:35:50.459+0000 1694f700 -1 osd.5 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.430415+0000 front 2019-12-09T15:34:20.674224+0000 (oldest deadline 2019-12-09T15:35:02.800267+0000) 2019-12-09T15:35:50.668 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:35:50.664+0000 1694f700 -1 osd.1 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.968085+0000 front 2019-12-09T15:34:18.827978+0000 (oldest deadline 2019-12-09T15:35:02.658673+0000) 2019-12-09T15:35:50.755 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:35:50.750+0000 1694f700 -1 osd.7 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.438646+0000 front 2019-12-09T15:34:24.700067+0000 (oldest deadline 2019-12-09T15:35:03.168626+0000) 2019-12-09T15:35:50.896 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:35:50.891+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 38sec e198 v198) 2019-12-09T15:35:50.911 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:35:50.907+0000 1694f700 -1 osd.0 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:24.436789+0000 front 2019-12-09T15:34:22.104761+0000 (oldest deadline 2019-12-09T15:35:05.507683+0000) 2019-12-09T15:35:50.924 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:35:50.920+0000 1694f700 -1 osd.2 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:25.339424+0000 front 2019-12-09T15:34:25.337961+0000 (oldest deadline 2019-12-09T15:35:10.634589+0000) 2019-12-09T15:35:51.064 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:35:51.060+0000 1694f700 -1 osd.3 198 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:34:22.106756+0000 front 2019-12-09T15:34:20.982141+0000 (oldest deadline 2019-12-09T15:35:06.229270+0000) 2019-12-09T15:36:29.183 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:36:29.139+0000 3aa81700 -1 osd.6 203 build_incremental_map_msg missing incremental map 203 2019-12-09T15:36:29.399 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:24:05.664 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.408 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:24:05.672 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.420 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:24:05.643 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.424 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:24:05.647 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.425 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:24:05.648 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.430 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:24:05.653 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.692 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:24:05.957 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.792 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:36:29.613+0000 3b282700 -1 osd.6 203 build_incremental_map_msg missing incremental map 203 2019-12-09T15:36:29.839 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:36:29.621+0000 3b282700 -1 osd.6 203 build_incremental_map_msg missing incremental map 203 2019-12-09T15:36:29.839 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:24:06.062 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.882 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:24:06.104 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:33.442 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:36:33.369+0000 36278700 -1 osd.6 204 build_incremental_map_msg missing incremental map 203 2019-12-09T15:36:47.979 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:47.973+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:48.517 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:48.510+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:48.952 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:48.948+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:49.519 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:49.512+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:49.872 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:36:49.864+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:36:49.921 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:49.913+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:50.122 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:36:50.117+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:36:50.250 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:50.242+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:36:50.487 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:50.481+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:50.837 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:36:50.830+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:36:50.900 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:50.893+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:51.164 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:36:51.160+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:36:51.212 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:51.208+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:36:51.477 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:51.471+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:51.847 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:36:51.840+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:36:51.874 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:51.868+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:52.143 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:36:52.139+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:36:52.237 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:52.232+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:36:52.241 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:52.237+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:36:52.495 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:52.489+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:52.855 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:36:52.848+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:36:52.863 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:52.856+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:53.129 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:36:53.124+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:36:53.175 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:36:53.170+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:36:53.213 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:53.205+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:36:53.213 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:53.208+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:36:53.509 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:53.502+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:53.831 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:36:53.824+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:36:53.914 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:53.907+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:54.128 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:36:54.121+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:36:54.176 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:36:54.170+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:36:54.193 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:54.187+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:36:54.194 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:54.190+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:36:54.530 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:54.523+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:54.833 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:36:54.828+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:36:54.952 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:54.947+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:55.168 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:55.162+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:36:55.169 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:55.165+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:36:55.174 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:36:55.168+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:36:55.228 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:36:55.221+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:36:55.555 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:55.549+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:55.872 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:36:55.865+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:36:55.946 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:55.939+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:56.144 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:36:56.139+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:36:56.198 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:56.191+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:36:56.198 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:56.194+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:36:56.218 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:36:56.213+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:36:56.610 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:56.604+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:56.867 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:36:56.860+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:36:56.962 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:56.955+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:57.119 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:36:57.114+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:36:57.193 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:36:57.188+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:36:57.216 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:57.210+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:36:57.216 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:57.213+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:36:57.626 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:57.620+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:57.888 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:36:57.881+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:36:57.986 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:57.978+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:58.109 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:36:58.104+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:36:58.201 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:36:58.195+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:36:58.252 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:58.248+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:36:58.255 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:58.252+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:36:58.409 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:36:58.404+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:36:58.664 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:58.658+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:58.910 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:36:58.904+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:36:58.964 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:58.956+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:36:59.118 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:36:59.113+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:36:59.180 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:36:59.175+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:36:59.300 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:59.296+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:36:59.301 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:36:59.297+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:36:59.485 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:36:59.466+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:36:59.723 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:36:59.715+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:36:59.882 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:36:59.876+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:36:59.920 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:36:59.914+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:00.131 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:00.125+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:00.157 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:00.152+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:00.314 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:00.309+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:00.315 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:00.311+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:00.494 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:00.476+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:00.741 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:00.735+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:00.885 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:37:00.879+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:37:00.932 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:00.927+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:01.128 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:01.123+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:01.129 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:01.125+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:01.270 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:01.265+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:01.271 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:01.267+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:01.490 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:01.477+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:01.701 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:01.694+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:01.921 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:37:01.916+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:37:01.929 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:01.923+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:02.098 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:02.093+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:02.125 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:02.120+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:02.268 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:02.263+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:02.269 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:02.264+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:02.484 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:02.465+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:02.669 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:02.662+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:02.877 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:37:02.871+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:37:02.911 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:02.904+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:03.114 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:03.108+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:03.155 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:03.151+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:03.247 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:03.243+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:03.249 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:03.246+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:03.473 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:03.462+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:03.662 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:03.655+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:03.865 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:37:03.859+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:37:03.906 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:03.900+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:04.168 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:04.163+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:04.169 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:04.163+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:04.263 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:04.257+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:04.263 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:04.260+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:04.497 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:04.480+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:04.622 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:04.616+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:04.848 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:37:04.842+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:37:04.911 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:04.904+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:05.127 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:05.121+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:05.185 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:05.179+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:05.240 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:05.233+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:05.241 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:05.236+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:05.468 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:05.444+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:05.482 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:05.463+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:05.670 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:05.664+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:05.848 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:37:05.841+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:37:05.877 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:05.871+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:06.094 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:06.089+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:06.179 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:06.174+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:06.268 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:06.263+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:06.269 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:06.265+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:06.449 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:06.424+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:06.462 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:06.444+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:06.704 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:06.697+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:06.841 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:06.834+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:06.899 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:37:06.893+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:37:07.055 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:07.049+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:07.170 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:07.164+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:07.229 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:07.224+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:07.232 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:07.227+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:07.463 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:07.441+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:07.476 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:07.458+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:07.717 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:07.711+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:07.825 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:07.818+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:07.893 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:37:07.887+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:37:08.033 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:08.027+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:08.191 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:08.183+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:08.256 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:08.248+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:08.257 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:08.252+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:08.450 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:08.434+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:08.487 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:08.469+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:08.759 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:08.752+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:08.829 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:08.824+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:08.861 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:37:08.853+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:36:48.995254+0000) 2019-12-09T15:37:09.031 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:09.025+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:09.241 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:09.236+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:09.286 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:09.280+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:09.288 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:09.284+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:09.431 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:09.424+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:09.464 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:09.444+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:09.731 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:09.725+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:09.868 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:09.860+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:10.038 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:10.032+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:10.212 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:10.205+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:10.273 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:10.266+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:10.276 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:10.272+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:10.478 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:10.454+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:10.492 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:10.473+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:10.694 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:10.686+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:10.851 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:10.844+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:11.032 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:11.027+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:11.215 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:11.209+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:11.250 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:11.244+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:11.251 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:11.247+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:11.451 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:11.430+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:11.465 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:11.447+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:11.719 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:11.712+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:11.806 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:11.799+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:12.041 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:12.037+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:12.203 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:12.198+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:12.207 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:12.200+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:12.208 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:12.204+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:12.458 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:12.434+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:12.472 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:12.454+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:12.756 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:12.748+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:12.800 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:12.792+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:13.037 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:13.031+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:13.208 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:13.202+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:13.209 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:13.205+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:13.238 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:13.233+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:13.437 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:13.419+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:13.446 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:13.433+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:13.793 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:13.786+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:13.844 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:13.837+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:14.035 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:14.030+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:14.254 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:14.247+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:14.254 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:14.250+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:14.258 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:14.255+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:14.394 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:14.387+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:14.426 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:14.408+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:14.800 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:14.794+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:14.850 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:14.844+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:15.001 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:14.995+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:15.220 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:15.215+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:15.222 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:15.217+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:15.223 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:15.219+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:15.405 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:15.399+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:15.437 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:15.419+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:15.788 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:15.782+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:15.867 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:15.861+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:15.979 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:15.973+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:16.217 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:16.212+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:16.219 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:16.215+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:16.268 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:16.263+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:16.431 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:16.425+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:16.473 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:16.454+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:16.790 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:16.784+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:16.862 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:16.857+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:17.001 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:16.998+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:17.223 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:17.218+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:17.224 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:17.220+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:17.317 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:17.313+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:17.475 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:17.470+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:17.503 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:17.486+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:17.821 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:17.814+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:17.906 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:17.900+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:17.993 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:17.988+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:18.199 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:18.189+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:18.201 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:18.196+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:18.369 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:18.365+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:18.453 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:18.447+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:18.485 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:18.467+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:18.840 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:18.833+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:18.941 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:18.934+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:18.975 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:18.971+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:19.190 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:19.185+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:19.194 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:19.189+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:19.401 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:19.398+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:19.499 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:19.494+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:19.538 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:19.509+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:19.844 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:19.838+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:19.946 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:19.941+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:19.960 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:19.955+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:20.153 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:20.147+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:20.157 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:20.153+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:20.404 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:20.399+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:20.520 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:20.500+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:20.538 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:20.517+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:20.845 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:20.840+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:20.930 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:20.926+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:20.970 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:20.965+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:21.130 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:21.123+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:21.131 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:21.127+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:21.249 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:37:21.243+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e204 v204) 2019-12-09T15:37:21.364 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:21.360+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:21.484 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:21.476+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:21.525 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:21.501+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:21.820 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:21.812+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:21.949 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:21.944+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:21.977 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:21.971+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:22.167 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:22.160+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:22.168 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:22.164+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:22.386 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:22.380+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:22.516 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:22.493+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:22.531 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:22.512+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:22.830 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:22.823+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:22.923 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:22.914+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:22.983 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:22.977+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:23.142 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:23.135+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:23.143 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:23.138+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:23.376 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:23.372+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:23.496 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:23.490+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:23.524 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:23.509+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:23.810 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:23.804+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:23.958 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:23.953+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:24.021 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:24.012+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:24.099 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:24.093+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:24.100 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:24.096+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:24.402 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:24.397+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:24.518 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:24.495+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:24.532 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:24.514+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:24.856 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:24.851+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:24.929 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:24.924+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:24.995 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:24.988+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:25.105 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:25.100+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:25.108 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:25.103+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:25.443 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:25.438+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:25.570 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:25.546+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:25.584 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:25.566+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:25.888 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:25.881+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:25.952 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:25.948+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:26.001 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:25.994+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:26.088 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:26.082+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:26.092 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:26.087+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:26.279 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:37:26.272+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e204 v204) 2019-12-09T15:37:26.427 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:26.421+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:26.544 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:26.539+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:26.574 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:26.558+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:26.933 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:26.926+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:26.970 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:26.963+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:26.983 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:26.978+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:27.078 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:27.074+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:27.082 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:27.078+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:27.426 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:27.420+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:27.568 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:27.540+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:27.585 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:27.564+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:27.933 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:27.926+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:28.000 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:27.994+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:28.009 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:28.004+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:28.103 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:28.099+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:28.107 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:28.103+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:28.428 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:28.423+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:28.600 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:28.568+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:28.613 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:28.596+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:28.933 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:28.926+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:29.000 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:28.994+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:29.026 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:29.022+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:29.134 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:29.128+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:29.135 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:29.131+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:29.430 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:29.426+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:29.603 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:29.596+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:29.637 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:29.618+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:29.933 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:29.926+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:30.011 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:30.006+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:30.056 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:30.050+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:30.155 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:30.149+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:30.156 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:30.152+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:30.408 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:30.404+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:30.620 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:30.597+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:30.633 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:30.616+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:30.929 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:30.921+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:31.016 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:31.012+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:31.026 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:31.019+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:31.172 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:31.166+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:31.173 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:31.169+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:31.306 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:37:31.296+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e204 v204) 2019-12-09T15:37:31.392 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:31.386+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:31.645 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:31.622+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:31.659 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:31.641+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:31.897 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:31.890+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:32.010 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:32.004+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:32.013 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:32.007+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:32.207 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:32.202+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:32.210 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:32.207+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:32.406 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:32.401+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:32.705 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:32.690+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:32.717 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:32.701+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:32.908 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:32.901+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:32.978 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:32.971+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:33.013 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:33.008+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:33.221 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:33.217+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:33.223 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:33.219+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:33.383 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:33.377+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:33.712 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:33.706+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:33.739 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:33.721+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:33.893 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:33.888+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:34.024 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:34.015+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:34.043 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:34.036+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:34.227 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:34.220+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:34.230 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:34.226+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:34.409 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:34.403+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:34.712 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:34.692+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:34.723 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:34.708+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:34.905 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:34.899+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:34.979 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:34.974+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:35.056 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:35.049+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:35.183 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:35.178+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:35.185 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:35.180+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:35.437 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:35.432+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:35.682 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:35.676+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:35.716 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:35.698+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:35.926 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:35.920+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:35.998 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:35.994+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:36.095 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:36.089+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:36.199 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:36.192+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:36.201 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:36.197+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:36.331 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:37:36.316+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e204 v204) 2019-12-09T15:37:36.483 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:36.477+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:36.704 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:36.698+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:36.727 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:36.713+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:36.946 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:36.939+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:37.020 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:37.015+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:37.060 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:37.053+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:37.223 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:37.217+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:37.224 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:37.220+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:37.488 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:37.483+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:37.689 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:37.675+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:37.696 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:37.685+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:37.920 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:37.913+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:38.022 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:38.017+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:38.036 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:38.029+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:38.235 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:38.231+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:38.239 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:38.235+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:38.489 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:38.483+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:38.734 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:38.728+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:38.762 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:38.747+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:38.958 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:38.953+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:39.020 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:39.013+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:39.042 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:39.036+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:39.284 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:39.275+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:39.285 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:39.280+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:39.446 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:39.440+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:39.751 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:39.728+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:39.765 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:39.746+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:40.015 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:40.002+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:40.047 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:40.043+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:40.068 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:40.062+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:40.288 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:40.283+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:40.292 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:40.288+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:40.460 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:40.454+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:40.763 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:40.743+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:40.777 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:40.760+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:40.999 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:40.993+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:41.096 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:41.091+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:41.111 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:41.105+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:41.333 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:41.327+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:41.335 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:41.330+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:41.346 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:37:41.343+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e204 v204) 2019-12-09T15:37:41.482 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:41.477+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:41.817 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:41.795+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:41.830 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:41.813+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:41.977 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:41.967+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:42.066 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:42.060+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:42.069 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:42.064+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:42.314 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:42.308+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:42.315 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:42.311+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:42.536 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:42.530+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:42.814 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:42.809+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:42.844 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:42.831+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:42.950 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:42.945+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:43.061 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:43.055+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:43.111 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:43.105+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:43.313 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:43.307+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:43.317 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:43.313+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:43.505 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:43.499+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:43.878 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:43.853+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:43.889 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:43.873+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:43.928 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:43.922+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:44.030 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:44.024+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:44.131 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:44.126+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:44.282 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:44.274+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:44.283 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:44.278+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:44.543 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:44.538+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:44.926 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:44.920+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:44.939 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:44.934+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:44.958 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:44.942+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:45.026 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:45.021+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:45.134 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:45.122+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:45.322 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:45.313+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:45.329 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:45.316+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:45.542 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:45.537+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:45.901 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:45.896+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:45.931 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:45.916+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:45.984 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:45.978+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:46.041 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:46.035+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:46.161 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:46.155+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:46.353 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:46.345+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:46.354 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:46.349+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:46.376 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:37:46.367+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e204 v204) 2019-12-09T15:37:46.579 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:46.575+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:46.906 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:46.901+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:46.941 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:46.924+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:46.947 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:46.943+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:47.016 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:47.010+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:47.214 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:47.208+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:47.379 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:47.373+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:47.385 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:47.380+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:47.575 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:47.571+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:47.970 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:47.965+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:47.995 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:47.989+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:48.003 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:47.987+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:48.011 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:48.005+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:48.208 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:48.203+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:48.425 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:48.418+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:48.427 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:48.422+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:48.607 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:48.600+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:49.007 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:48.979+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:49.013 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:49.008+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:49.026 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:48.999+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:49.066 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:49.062+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:49.236 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:49.230+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:49.411 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:49.405+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:49.413 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:49.408+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:49.625 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:49.620+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:50.001 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:49.997+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:50.008 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:49.977+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:50.025 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:50.005+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 251 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:50.083 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:50.079+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:50.226 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:50.218+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:50.278 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:37:50.273+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:37:49.951503+0000) 2019-12-09T15:37:50.379 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:50.375+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:50.383 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:50.379+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:50.651 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:37:50.645+0000 1694f700 -1 osd.3 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.618646+0000 front 2019-12-09T15:36:06.994690+0000 (oldest deadline 2019-12-09T15:36:50.088285+0000) 2019-12-09T15:37:51.000 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:50.967+0000 1694f700 -1 osd.6 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:36:24.853071+0000 (oldest deadline 2019-12-09T15:37:04.853071+0000) 2019-12-09T15:37:51.014 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:50.997+0000 1694f700 -1 osd.6 204 get_health_metrics reporting 288 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:51.030 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:37:51.025+0000 1694f700 -1 osd.4 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:05.453913+0000 front 2019-12-09T15:36:05.432770+0000 (oldest deadline 2019-12-09T15:36:48.294129+0000) 2019-12-09T15:37:51.037 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:37:51.032+0000 1694f700 -1 osd.2 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:08.264423+0000 front 2019-12-09T15:36:08.226464+0000 (oldest deadline 2019-12-09T15:36:53.016676+0000) 2019-12-09T15:37:51.238 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:37:51.234+0000 1694f700 -1 osd.5 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:02.696711+0000 front 2019-12-09T15:36:08.263740+0000 (oldest deadline 2019-12-09T15:36:47.328291+0000) 2019-12-09T15:37:51.313 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:37:51.308+0000 1694f700 -1 osd.7 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:04.383570+0000 front 2019-12-09T15:36:04.404733+0000 (oldest deadline 2019-12-09T15:37:49.951503+0000) 2019-12-09T15:37:51.431 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:51.427+0000 1694f700 -1 osd.0 204 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:36:06.857946+0000 front 2019-12-09T15:36:06.625264+0000 (oldest deadline 2019-12-09T15:36:49.579681+0000) 2019-12-09T15:37:51.433 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:51.430+0000 1694f700 -1 osd.0 204 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:51.434 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:37:51.419+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e204 v204) 2019-12-09T15:37:52.236 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:52.162+0000 3a280700 -1 osd.6 205 build_incremental_map_msg missing incremental map 203 2019-12-09T15:37:52.390 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:37:52.324+0000 1694f700 -1 osd.6 205 get_health_metrics reporting 251 slow ops, oldest is osd_op(client.4355.0:112607 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e202) 2019-12-09T15:37:52.480 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:52.472+0000 1694f700 -1 osd.0 205 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:113871 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e200) 2019-12-09T15:37:53.705 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:37:53.664+0000 1694f700 -1 osd.0 206 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:113921 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5467864320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e204) 2019-12-09T15:37:54.001 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T15:37:53.987+0000 20e25700 -1 RGWWatcher::handle_error cookie 5467864320 err (107) Transport endpoint is not connected 2019-12-09T15:38:01.876 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:25:38.140 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:01.887 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:25:38.151 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:01.963 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:25:38.185 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:01.964 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:25:38.187 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:02.040 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:25:38.262 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:02.046 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:25:38.268 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:02.077 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:25:38.299 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:02.145 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:25:38.409 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:02.180 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:25:38.402 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:27.355 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:26:03.619 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:27.356 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:26:03.621 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:27.404 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:26:03.669 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:27.405 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:26:03.627 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:27.406 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:26:03.629 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:27.435 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:26:03.657 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:27.440 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:26:03.663 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:27.461 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:26:03.683 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:38:27.494 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:26:03.716 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:39:15.100 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:15.095+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:15.165 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:15.158+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:15.590 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:15.586+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:16.076 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:16.071+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:16.215 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:16.210+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:16.505 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:16.500+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:16.630 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:16.625+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:17.036 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:17.032+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:17.152 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:17.147+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:17.203 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:17.197+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:17.544 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:17.537+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:17.616 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:17.611+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:18.011 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:18.005+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:18.206 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:18.201+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:18.220 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:18.213+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:18.547 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:18.541+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:18.621 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:18.615+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:18.966 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:18.961+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:19.195 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:19.190+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:19.212 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:19.206+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:19.534 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:19.529+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:19.576 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:19.572+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:19.673 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:19.668+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:19.945 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:19.941+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:20.205 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:20.200+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:20.230 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:20.223+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:20.530 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:20.526+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:20.585 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:20.579+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:20.637 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:20.631+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:20.996 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:20.992+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:21.190 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:21.184+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:21.205 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:21.201+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:21.519 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:21.515+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:21.549 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:21.545+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:21.637 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:21.629+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:22.046 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:22.041+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:22.185 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:22.181+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:22.211 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:22.205+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:22.480 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:22.476+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:22.550 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:22.546+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:22.681 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:22.675+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:23.011 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:23.006+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:23.231 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:23.226+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:23.243 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:23.238+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:23.488 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:23.484+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:23.526 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:23.522+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:23.644 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:23.639+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:24.055 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:24.050+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:24.204 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:24.198+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:24.276 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:24.272+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:24.497 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:24.492+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:24.532 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:24.528+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:24.619 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:24.612+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:25.071 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:25.067+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:25.206 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:25.201+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:25.319 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:25.313+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:25.505 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:25.502+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:25.509 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:25.502+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:25.661 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:25.656+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:26.027 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:26.023+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:26.213 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:26.207+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:26.327 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:26.322+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:26.473 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:26.468+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:26.541 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:26.537+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:26.682 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:26.677+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:26.992 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:26.988+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:27.243 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:27.239+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:27.359 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:27.351+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:27.519 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:27.514+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:27.565 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:27.560+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:27.737 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:27.732+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:27.963 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:27.958+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:28.245 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:28.240+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:28.353 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:28.348+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:28.509 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:28.504+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:28.576 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:28.571+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:28.696 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:28.691+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:28.927 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:28.923+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:29.225 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:29.220+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:29.379 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:29.374+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:29.548 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:29.541+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:29.568 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:29.564+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:29.722 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:29.717+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:29.958 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:29.954+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:30.253 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:30.248+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:30.352 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:30.347+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:30.511 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:30.507+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:30.582 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:30.578+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:30.685 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:30.680+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:30.978 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:30.974+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:31.234 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:31.228+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:31.315 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:31.309+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:31.519 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:31.503+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:31.580 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:31.575+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:31.663 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:31.656+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:31.962 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:31.957+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:32.207 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:32.202+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:32.339 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:32.333+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:32.514 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:32.508+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:32.619 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:32.613+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:32.680 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:32.674+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:32.932 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:32.926+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:33.206 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:33.200+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:33.311 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:33.305+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:33.480 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:33.473+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:33.620 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:33.615+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:33.645 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:33.639+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:33.917 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:33.907+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:34.204 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:34.198+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:34.301 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:34.295+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:34.444 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:34.435+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:34.659 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:34.653+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:34.668 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:34.663+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:34.869 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:34.865+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:35.178 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:35.173+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:35.270 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:35.265+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:35.427 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:35.422+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:35.654 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:35.648+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:35.696 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:35.691+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:35.917 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:35.912+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:36.178 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:36.173+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:36.262 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:36.257+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:36.395 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:36.390+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:36.690 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:36.684+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:36.713 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:36.708+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:36.970 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:36.965+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:37.192 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:37.186+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:37.247 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:37.242+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:37.379 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:37.374+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:37.652 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:37.646+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:37.750 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:37.745+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:38.015 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:38.010+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:38.172 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:38.167+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:38.232 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:38.227+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:38.381 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:38.377+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:38.665 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:38.659+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:38.759 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:38.754+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:39.055 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:39.050+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:39.158 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:39.153+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:39.249 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:39.244+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:39.387 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:39.380+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:39.633 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:39.626+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:39.778 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:39.774+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:40.010 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:40.005+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:40.149 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:40.143+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:40.266 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:40.261+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:40.408 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:40.404+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:40.624 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:40.614+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:40.830 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:40.825+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:40.992 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:40.988+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:41.186 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:41.181+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:41.284 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:41.280+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:41.457 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:41.453+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:41.677 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:41.671+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:41.817 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:41.812+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:41.979 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:41.975+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:42.176 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:42.169+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:42.324 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:42.319+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:42.469 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:42.465+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:42.650 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:42.645+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:42.819 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:42.815+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:42.991 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:42.987+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:43.136 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:43.131+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:43.296 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:43.290+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:43.478 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:43.474+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:43.662 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:43.656+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:43.860 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:43.855+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:44.019 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:44.014+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:44.110 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:44.105+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:44.290 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:44.286+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:44.465 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:44.460+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:44.667 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:44.661+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:44.828 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:44.824+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:45.057 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:45.052+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:45.155 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:45.150+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:45.322 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:45.317+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:45.495 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:45.491+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:45.648 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:45.642+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:45.794 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:45.789+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:46.101 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:46.095+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:46.144 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:46.138+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:46.303 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:46.298+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:46.497 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:46.492+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:46.618 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:46.612+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:46.845 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:46.840+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:47.142 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:47.137+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:47.195 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:47.189+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:47.294 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:47.288+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:47.379 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:39:47.374+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 42sec e210 v210) 2019-12-09T15:39:47.494 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:47.490+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:47.673 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:47.668+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:47.833 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:47.828+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:48.156 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:48.150+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:48.190 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:48.186+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:48.257 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:48.252+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:48.535 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:48.531+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:48.647 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:48.639+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:48.800 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:48.796+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:49.188 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:49.182+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:49.232 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:49.227+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:49.277 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:49.273+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:49.522 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:49.517+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:49.625 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:49.619+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:49.797 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:49.792+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:50.210 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:50.204+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:50.246 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:50.241+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:50.267 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:50.262+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:50.555 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:50.550+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:50.663 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:50.657+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:50.847 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:50.842+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:51.168 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:51.162+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:51.222 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:51.218+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:51.256 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:51.252+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:51.577 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:51.572+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:51.664 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:51.658+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:51.810 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:51.806+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:52.129 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:52.123+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:52.252 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:52.247+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:52.301 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:52.297+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:52.399 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:39:52.394+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 42sec e210 v210) 2019-12-09T15:39:52.598 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:52.595+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:52.642 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:52.637+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:52.734 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:39:52.728+0000 1694f700 -1 osd.7 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.058376+0000 front 2019-12-09T15:38:34.711401+0000 (oldest deadline 2019-12-09T15:39:52.604982+0000) 2019-12-09T15:39:52.788 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:52.783+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:53.142 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:53.137+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:53.242 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:53.237+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:53.296 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:53.291+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:53.612 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:53.607+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:53.645 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:53.640+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:53.738 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:39:53.731+0000 1694f700 -1 osd.7 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.058376+0000 front 2019-12-09T15:38:34.711401+0000 (oldest deadline 2019-12-09T15:39:52.604982+0000) 2019-12-09T15:39:53.790 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:53.786+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:54.146 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:54.140+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:54.235 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:54.230+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:54.254 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:54.249+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:54.604 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:54.600+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:54.624 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:54.618+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:54.727 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:39:54.722+0000 1694f700 -1 osd.7 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.058376+0000 front 2019-12-09T15:38:34.711401+0000 (oldest deadline 2019-12-09T15:39:52.604982+0000) 2019-12-09T15:39:54.843 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:54.838+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:55.187 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:55.181+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:55.224 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:55.219+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:55.236 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:55.229+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:55.637 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:55.633+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:55.651 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:55.646+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:55.773 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:39:55.767+0000 1694f700 -1 osd.7 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.058376+0000 front 2019-12-09T15:38:34.711401+0000 (oldest deadline 2019-12-09T15:39:52.604982+0000) 2019-12-09T15:39:55.799 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:55.794+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:56.171 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:56.166+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:56.215 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:56.210+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:56.221 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:56.217+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:56.613 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:56.607+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:56.622 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:56.618+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:56.800 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:56.795+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:56.817 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:39:56.812+0000 1694f700 -1 osd.7 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.058376+0000 front 2019-12-09T15:38:34.711401+0000 (oldest deadline 2019-12-09T15:39:52.604982+0000) 2019-12-09T15:39:57.178 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:57.172+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:57.221 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:57.217+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:57.231 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:57.228+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:57.419 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:39:57.414+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 42sec e210 v210) 2019-12-09T15:39:57.594 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:57.590+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:57.641 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:57.635+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:57.801 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:57.797+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:57.811 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:39:57.807+0000 1694f700 -1 osd.7 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.058376+0000 front 2019-12-09T15:38:34.711401+0000 (oldest deadline 2019-12-09T15:39:52.604982+0000) 2019-12-09T15:39:58.199 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:58.193+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:58.206 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:58.201+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:58.234 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:58.230+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:58.632 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:58.627+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:58.649 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:58.643+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:58.784 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:39:58.778+0000 1694f700 -1 osd.7 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.058376+0000 front 2019-12-09T15:38:34.711401+0000 (oldest deadline 2019-12-09T15:39:52.604982+0000) 2019-12-09T15:39:58.821 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:58.817+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:39:59.163 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:39:59.157+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:39:59.212 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:39:59.208+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:39:59.273 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:39:59.269+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:39:59.613 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:39:59.608+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:39:59.660 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:39:59.654+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:39:59.774 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:39:59.768+0000 1694f700 -1 osd.7 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.058376+0000 front 2019-12-09T15:38:34.711401+0000 (oldest deadline 2019-12-09T15:39:52.604982+0000) 2019-12-09T15:39:59.814 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:39:59.809+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:40:00.174 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:40:00.168+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:40:00.218 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:40:00.214+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:40:00.244 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:40:00.240+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:40:00.663 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:40:00.659+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:40:00.687 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:40:00.681+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:40:00.818 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:40:00.812+0000 1694f700 -1 osd.7 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.058376+0000 front 2019-12-09T15:38:34.711401+0000 (oldest deadline 2019-12-09T15:39:52.604982+0000) 2019-12-09T15:40:00.853 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:40:00.849+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:40:01.201 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:40:01.197+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:40:01.225 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:40:01.219+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:40:01.254 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:40:01.250+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:40:01.662 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:40:01.657+0000 1694f700 -1 osd.0 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.219653+0000 front 2019-12-09T15:38:32.418645+0000 (oldest deadline 2019-12-09T15:39:15.575937+0000) 2019-12-09T15:40:01.687 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:40:01.681+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:40:01.778 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:40:01.772+0000 1694f700 -1 osd.7 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.058376+0000 front 2019-12-09T15:38:34.711401+0000 (oldest deadline 2019-12-09T15:39:52.604982+0000) 2019-12-09T15:40:01.862 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:40:01.857+0000 1694f700 -1 osd.3 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.276362+0000 front 2019-12-09T15:38:35.149342+0000 (oldest deadline 2019-12-09T15:39:15.106944+0000) 2019-12-09T15:40:02.189 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:40:02.184+0000 1694f700 -1 osd.1 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:32.423463+0000 front 2019-12-09T15:38:37.277159+0000 (oldest deadline 2019-12-09T15:39:14.912523+0000) 2019-12-09T15:40:02.262 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:40:02.257+0000 1694f700 -1 osd.4 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.506536+0000 front 2019-12-09T15:38:32.191205+0000 (oldest deadline 2019-12-09T15:39:14.304507+0000) 2019-12-09T15:40:02.279 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:40:02.274+0000 1694f700 -1 osd.2 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:34.713033+0000 front 2019-12-09T15:38:34.713407+0000 (oldest deadline 2019-12-09T15:39:16.781161+0000) 2019-12-09T15:40:02.511 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:40:02.501+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 42sec e210 v210) 2019-12-09T15:40:02.671 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:40:02.666+0000 1694f700 -1 osd.5 210 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:38:37.274439+0000 front 2019-12-09T15:38:37.309675+0000 (oldest deadline 2019-12-09T15:39:18.670927+0000) 2019-12-09T15:40:27.113 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:28:03.377 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:27.128 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:28:03.393 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:27.129 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:28:03.352 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:27.135 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:28:03.357 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:27.136 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:28:03.359 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:27.141 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:28:03.363 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:27.289 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:28:03.511 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:27.300 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:28:03.522 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:27.362 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:28:03.627 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:58.224 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:40:58.220+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:40:59.215 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:40:59.212+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:00.172 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:00.168+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:00.792 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:00.787+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:01.196 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:01.190+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:01.199 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:01.195+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:01.637 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:01.628+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:01.827 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:01.820+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:01.831 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:01.825+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:01.956 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:01.952+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:02.183 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:02.177+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:02.184 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:02.180+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:02.660 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:02.655+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:02.800 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:02.794+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:02.806 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:02.801+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:02.815 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:02.809+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:02.923 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:02.917+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:03.201 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:03.195+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:03.203 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:03.198+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:03.674 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:03.669+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:03.790 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:03.782+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:03.823 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:03.815+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:03.850 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:03.846+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:03.976 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:03.969+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:04.198 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:04.193+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:04.199 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:04.195+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:04.683 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:04.677+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:04.793 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:04.786+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:04.857 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:04.851+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:04.873 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:04.867+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:04.991 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:04.986+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:05.170 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:05.165+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:05.171 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:05.167+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:05.656 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:05.649+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:05.769 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:05.762+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:05.857 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:05.848+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:05.885 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:05.879+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:05.995 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:05.989+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:06.217 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:06.212+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:06.218 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:06.214+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:06.668 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:06.662+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:06.798 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:06.792+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:06.892 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:06.884+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:06.911 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:06.905+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:06.963 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:06.956+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:07.229 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:07.221+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:07.229 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:07.224+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:07.692 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:07.686+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:07.797 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:07.790+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:07.851 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:07.845+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:07.929 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:07.922+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:07.951 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:07.945+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:08.279 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:08.272+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:08.284 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:08.279+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:08.725 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:08.717+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:08.822 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:08.815+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:08.845 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:08.838+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:08.912 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:08.901+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:08.945 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:08.938+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:09.318 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:09.311+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:09.321 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:09.316+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:09.731 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:09.725+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:09.831 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:09.824+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:09.838 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:09.831+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:09.888 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:09.882+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:09.955 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:09.949+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:10.348 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:10.343+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:10.349 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:10.345+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:10.782 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:10.777+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:10.815 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:10.809+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:10.858 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:10.851+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:10.876 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:10.870+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:10.889 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:10.883+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 18 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:10.949 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:10.942+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:11.394 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:11.388+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:11.395 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:11.391+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:11.775 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:11.769+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:11.830 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:11.825+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:11.850 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:11.844+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:11.859 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:11.854+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:11.882 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:11.875+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 36 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:11.931 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:11.926+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:12.417 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:12.410+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:12.418 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:12.412+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:12.825 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:12.820+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:12.857 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:12.851+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:12.872 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:12.867+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:12.889 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:12.882+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:12.943 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:12.936+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 40 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:12.985 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:12.978+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:13.390 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:13.385+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:13.393 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:13.389+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:13.859 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:13.853+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:13.866 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:13.859+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:13.884 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:13.877+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:13.912 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:13.907+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:13.956 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:13.950+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:14.034 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:14.017+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 151 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:14.426 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:14.419+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:14.427 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:14.423+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:14.821 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:14.816+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:14.910 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:14.904+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:14.935 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:14.923+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:14.936 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:14.929+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:14.938 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:14.933+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:15.020 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:15.003+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:15.453 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:15.446+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:15.454 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:15.450+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:15.871 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:15.864+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:15.883 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:15.879+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:15.896 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:15.890+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:15.913 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:15.907+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:15.952 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:15.948+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:16.060 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:16.039+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:16.476 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:16.470+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:16.476 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:16.473+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:16.874 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:16.867+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:16.891 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:16.884+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:16.894 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:16.887+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:16.907 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:16.900+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:16.947 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:16.943+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:17.068 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:17.058+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:17.094 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:17.076+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:17.486 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:17.481+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:17.489 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:17.485+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:17.862 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:17.855+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:17.911 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:17.904+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:17.920 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:17.915+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:17.921 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:17.915+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:17.935 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:17.928+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:18.060 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:18.054+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:18.096 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:18.084+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:18.507 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:18.501+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:18.511 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:18.507+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:18.824 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:18.817+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:18.894 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:18.890+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:18.900 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:18.895+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:18.915 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:18.909+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:18.959 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:18.953+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:19.108 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:19.078+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:19.127 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:19.103+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:19.476 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:19.470+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:19.476 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:19.473+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:19.859 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:19.854+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:19.879 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:19.873+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:19.880 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:19.873+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:19.928 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:19.920+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:19.972 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:19.965+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:20.134 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:20.108+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:20.147 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:20.130+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:20.507 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:20.502+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:20.510 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:20.506+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:20.824 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:20.817+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:20.884 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:20.878+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:20.923 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:20.917+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:20.930 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:20.924+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:20.980 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:20.974+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:21.122 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:21.116+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:21.161 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:21.139+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:21.529 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:21.524+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:21.530 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:21.526+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:21.826 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:21.817+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:21.915 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:21.903+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:21.933 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:21.927+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:21.936 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:21.932+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:21.983 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:21.973+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:22.105 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:22.099+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:22.144 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:22.123+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:22.568 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:22.561+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:22.570 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:22.565+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:22.811 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:22.806+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:22.908 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:22.901+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:22.929 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:22.922+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:22.966 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:22.958+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:22.973 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:22.966+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:23.136 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:23.110+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:23.151 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:23.132+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:23.553 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:23.546+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:23.554 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:23.550+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:23.778 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:23.773+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:23.921 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:23.914+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:23.949 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:23.943+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:23.969 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:23.963+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:24.007 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:23.999+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:24.171 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:24.164+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:24.193 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:24.177+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:24.512 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:24.506+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:24.513 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:24.509+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:24.752 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:24.746+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:24.901 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:24.890+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:24.969 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:24.963+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:24.975 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:24.968+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:24.986 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:24.981+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:25.231 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:25.203+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:25.247 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:25.226+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:25.518 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:25.511+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:25.520 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:25.516+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:25.759 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:25.754+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:25.951 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:25.943+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:25.970 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:25.964+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:25.971 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:25.964+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:26.040 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:26.035+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:26.233 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:26.207+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:26.252 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:26.229+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:26.522 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:26.511+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:26.523 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:26.515+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:26.722 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:26.717+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:26.947 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:26.940+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:26.961 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:26.955+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:27.005 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:27.000+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:27.045 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:27.039+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:27.240 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:27.235+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:27.278 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:27.258+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:27.523 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:27.515+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:27.524 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:27.520+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:27.727 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:27.722+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:27.913 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:27.907+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:27.940 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:27.935+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:27.960 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:27.955+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:28.009 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:28.003+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:28.320 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:28.295+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:28.336 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:28.317+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:28.525 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:28.520+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:28.529 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:28.525+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:28.763 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:28.758+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:28.895 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:28.890+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:28.964 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:28.959+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:28.967 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:28.962+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:29.058 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:29.054+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:29.279 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:29.275+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:29.296 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:29.285+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:29.583 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:29.576+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:29.584 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:29.580+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:29.817 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:29.809+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:29.866 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:29.860+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:29.980 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:29.974+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:30.022 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:30.015+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:30.067 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:30.062+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:30.275 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:30.255+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:30.291 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:30.272+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:30.569 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:30.560+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:30.571 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:30.563+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:30.822 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:30.816+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:30.841 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:30.830+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:30.937 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:30.931+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:30.981 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:30.976+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:31.071 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:31.063+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:31.342 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:31.317+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:31.363 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:31.338+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:31.555 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:31.551+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:31.559 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:31.555+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:31.811 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:31.806+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:31.842 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:31.835+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:31.967 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:31.960+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:31.979 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:31.973+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:32.100 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:32.095+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:32.334 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:32.300+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:32.349 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:32.328+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:32.572 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:32.567+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:32.575 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:32.572+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:32.766 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:32.761+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:32.877 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:32.872+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:32.962 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:32.958+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:33.014 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:33.010+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:33.057 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:33.053+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:33.060 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:41:33.054+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e216 v216) 2019-12-09T15:41:33.338 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:33.312+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:33.354 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:33.335+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:33.560 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:33.554+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:33.562 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:33.557+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:33.800 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:33.794+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:33.919 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:33.914+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:33.952 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:33.940+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:34.025 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:34.019+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:34.034 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:34.030+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:34.344 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:34.339+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:34.378 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:34.362+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:34.529 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:34.524+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:34.533 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:34.529+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:34.770 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:34.764+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:34.919 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:34.913+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:34.936 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:34.929+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:34.994 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:34.989+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:35.076 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:35.070+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:35.343 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:35.317+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:35.361 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:35.340+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:35.551 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:35.544+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:35.553 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:35.548+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:35.800 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:35.794+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:35.893 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:35.887+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:35.921 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:35.915+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:36.035 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:36.029+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:36.106 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:36.100+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:36.398 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:36.370+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:36.417 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:36.395+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:36.538 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:36.531+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:36.539 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:36.535+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:36.826 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:36.821+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:36.859 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:36.854+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:36.900 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:36.895+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:37.030 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:37.025+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:37.159 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:37.146+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:37.452 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:37.447+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:37.492 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:37.472+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:37.496 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:37.490+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:37.497 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:37.493+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:37.854 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:37.849+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:37.855 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:37.850+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:37.873 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:37.867+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:38.021 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:38.015+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:38.085 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:41:38.077+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e216 v216) 2019-12-09T15:41:38.141 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:38.135+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:38.482 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:38.456+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:38.499 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:38.478+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:38.521 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:38.514+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:38.526 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:38.518+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:38.819 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:38.815+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:38.872 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:38.860+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:38.886 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:38.881+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:38.977 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:38.972+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:39.163 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:39.157+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:39.477 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:39.453+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:39.477 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:39.471+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:39.478 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:39.474+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:39.493 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:39.473+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:39.794 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:39.789+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:39.868 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:39.863+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:39.894 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:39.889+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:40.026 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:40.020+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:40.175 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:40.169+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:40.461 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:40.435+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:40.481 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:40.458+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:40.490 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:40.483+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:40.492 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:40.487+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:40.827 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:40.822+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:40.889 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:40.884+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:40.926 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:40.920+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:41.034 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:41.028+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:41.140 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:41.134+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:41.461 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:41.435+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:41.471 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:41.465+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:41.473 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:41.468+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:41.478 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:41.457+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:41.874 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:41.869+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:41.920 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:41.914+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:41.967 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:41.961+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:41.999 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:41.994+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:42.144 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:42.138+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:42.452 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:42.447+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:42.455 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:42.449+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:42.530 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:42.504+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:42.548 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:42.526+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:42.900 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:42.895+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:42.964 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:42.956+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:42.979 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:42.974+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:43.028 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:43.024+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:43.093 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:41:43.089+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e216 v216) 2019-12-09T15:41:43.111 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:43.104+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:43.476 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:43.470+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:43.477 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:43.473+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:43.584 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:43.578+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:43.623 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:43.603+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:43.864 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:43.853+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:43.921 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:43.915+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:43.964 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:43.957+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:44.052 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:44.046+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:44.095 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:44.087+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:44.485 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:44.479+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:44.492 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:44.486+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:44.603 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:44.596+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:44.648 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:44.637+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:44.851 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:44.845+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:44.944 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:44.937+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:44.962 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:44.955+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:45.104 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:45.093+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:45.148 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:45.142+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:45.524 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:45.518+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:45.525 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:45.521+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:45.656 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:45.631+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:45.673 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:45.652+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:45.881 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:45.875+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:45.995 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:45.988+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:46.009 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:45.999+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:46.055 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:46.050+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:46.202 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:46.196+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:46.513 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:46.507+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:46.514 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:46.510+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:46.622 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:46.616+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:46.660 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:46.637+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:46.898 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:46.893+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:46.991 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:46.986+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:47.005 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:46.998+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:47.070 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:47.065+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:47.210 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:47.203+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:47.511 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:47.506+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:47.517 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:47.512+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:47.621 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:47.615+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:47.662 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:47.643+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:47.898 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:47.893+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:48.030 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:48.024+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:48.060 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:48.053+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:48.081 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:48.076+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:48.122 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:41:48.114+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e216 v216) 2019-12-09T15:41:48.250 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:48.244+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:48.571 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:48.562+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:48.573 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:48.568+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:48.685 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:48.654+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:48.701 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:48.676+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:48.931 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:48.925+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:49.028 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:49.021+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:49.041 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:49.033+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:49.041 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:49.037+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:49.217 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:49.207+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:49.554 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:49.550+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:49.556 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:49.553+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:49.674 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:49.647+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:49.692 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:49.669+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:49.953 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:49.948+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:50.026 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:50.021+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:50.061 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:50.051+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:50.088 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:50.082+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:50.258 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:50.252+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:50.568 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:50.563+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:50.571 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:50.567+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:50.652 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:50.645+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:50.686 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:50.667+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:50.943 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:50.938+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:51.038 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:51.031+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:51.051 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:51.044+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:51.070 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:51.064+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:51.231 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:51.225+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:51.575 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:51.568+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:51.577 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:51.572+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:51.680 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:51.675+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:51.709 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:51.693+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:51.963 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:51.959+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:52.026 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:52.019+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:52.028 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:52.022+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:52.041 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:52.037+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:52.250 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:52.243+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:52.542 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:52.536+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:52.544 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:52.539+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:52.758 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:52.730+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:52.772 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:52.751+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:52.986 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:52.981+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:52.987 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:52.981+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:53.040 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:53.033+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:53.063 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:53.059+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:53.145 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:41:53.136+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e216 v216) 2019-12-09T15:41:53.261 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:53.255+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:53.500 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:53.494+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:53.502 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:53.498+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:53.777 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:53.751+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:53.793 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:53.773+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:53.946 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:53.942+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:54.035 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:54.029+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:54.060 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:54.054+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:54.064 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:54.059+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:54.296 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:54.290+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:54.528 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:54.523+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:54.537 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:54.528+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:54.753 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:54.726+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:54.779 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:54.749+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:54.969 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:54.963+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:55.041 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:55.032+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:55.042 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:55.034+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:55.081 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:55.074+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:55.300 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:55.294+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:55.532 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:55.523+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:55.534 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:55.528+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:55.763 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:55.750+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:55.795 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:55.775+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:56.019 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:56.011+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:56.019 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:56.012+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:56.025 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:56.018+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:56.057 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:56.052+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:56.349 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:56.343+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:56.570 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:56.562+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:56.571 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:56.566+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:56.823 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:56.797+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:56.840 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:56.819+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:57.000 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:56.995+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:57.038 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:57.032+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:57.054 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:57.048+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:57.061 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:57.054+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:57.381 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:57.375+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:57.564 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:57.557+0000 1694f700 -1 osd.0 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:14.921858+0000 front 2019-12-09T15:40:14.933860+0000 (oldest deadline 2019-12-09T15:41:00.224408+0000) 2019-12-09T15:41:57.566 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:57.560+0000 1694f700 -1 osd.0 216 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:41:57.806 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:57.778+0000 1694f700 -1 osd.6 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:40:36.773192+0000 (oldest deadline 2019-12-09T15:41:16.773192+0000) 2019-12-09T15:41:57.824 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:57.801+0000 1694f700 -1 osd.6 216 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:58.011 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:41:58.004+0000 1694f700 -1 osd.7 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.596686+0000 front 2019-12-09T15:40:18.505426+0000 (oldest deadline 2019-12-09T15:41:01.233388+0000) 2019-12-09T15:41:58.056 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:41:58.049+0000 1694f700 -1 osd.3 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:15.547486+0000 front 2019-12-09T15:40:14.863434+0000 (oldest deadline 2019-12-09T15:41:00.740754+0000) 2019-12-09T15:41:58.088 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:41:58.082+0000 1694f700 -1 osd.2 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:18.610317+0000 front 2019-12-09T15:40:18.611543+0000 (oldest deadline 2019-12-09T15:41:01.398565+0000) 2019-12-09T15:41:58.099 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:41:58.092+0000 1694f700 -1 osd.5 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:20.282075+0000 front 2019-12-09T15:40:17.718257+0000 (oldest deadline 2019-12-09T15:41:02.416947+0000) 2019-12-09T15:41:58.219 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:41:58.206+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e216 v216) 2019-12-09T15:41:58.397 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:41:58.393+0000 1694f700 -1 osd.4 216 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:40:17.693832+0000 front 2019-12-09T15:40:17.306293+0000 (oldest deadline 2019-12-09T15:41:01.261384+0000) 2019-12-09T15:41:58.960 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:41:58.923+0000 1694f700 -1 osd.6 217 get_health_metrics reporting 204 slow ops, oldest is osd_op(client.4355.0:111665 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e214) 2019-12-09T15:41:59.089 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:41:58.835+0000 1694f700 -1 osd.0 217 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:114257 5.5 5:a93a5511:::notify.2:head [watch ping cookie 823419904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T15:42:00.688 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T15:42:00.669+0000 20e25700 -1 RGWWatcher::handle_error cookie 823419904 err (107) Transport endpoint is not connected 2019-12-09T15:43:29.849 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:29.844+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:30.142 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:30.137+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:30.817 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:30.812+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:31.183 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:31.177+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:31.845 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:31.838+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:31.987 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:31.980+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:32.107 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:32.102+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:32.202 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:32.197+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:32.894 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:32.888+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:33.031 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:33.025+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:33.121 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:33.115+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:33.239 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:33.233+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:33.334 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:33.329+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:33.823 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:33.817+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:33.913 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:33.907+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:34.063 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:34.057+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:34.176 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:34.170+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:34.252 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:34.246+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:34.349 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:34.342+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:34.843 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:34.839+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:34.945 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:34.940+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:35.091 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:35.087+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:35.191 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:35.185+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:35.260 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:35.256+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:35.329 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:35.323+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:35.735 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:35.730+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:35.872 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:35.868+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:35.999 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:35.993+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:36.104 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:36.099+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:36.246 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:36.240+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:36.289 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:36.285+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:36.345 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:36.338+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:36.702 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:36.697+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:36.856 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:36.851+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:36.968 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:36.964+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:37.063 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:37.059+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:37.257 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:37.252+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:37.280 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:37.274+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:37.383 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:37.378+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:37.713 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:37.703+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:37.822 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:37.817+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:37.999 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:37.993+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:38.115 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:38.111+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:38.237 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:38.232+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:38.255 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:38.250+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:38.407 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:38.401+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:38.734 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:38.729+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:38.787 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:38.781+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:39.010 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:39.003+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:39.167 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:39.162+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:39.291 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:39.285+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:39.311 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:39.305+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:39.440 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:39.434+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:39.739 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:39.735+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:39.831 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:39.825+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:40.018 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:40.012+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:40.167 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:40.162+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:40.278 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:40.271+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:40.281 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:40.276+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:40.442 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:40.436+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:40.726 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:40.721+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:40.804 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:40.799+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:41.033 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:41.028+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:41.144 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:41.140+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:41.279 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:41.273+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:41.331 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:41.327+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:41.439 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:41.433+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:41.776 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:41.771+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:41.800 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:41.796+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:42.045 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:42.039+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:42.156 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:42.152+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:42.244 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:42.240+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:42.354 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:42.349+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:42.434 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:42.428+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:42.816 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:42.812+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:42.826 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:42.822+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:43.057 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:43.051+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:43.153 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:43.148+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:43.239 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:43.235+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:43.369 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:43.362+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:43.402 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:43.396+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:43.807 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:43.802+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:43.854 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:43.849+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:44.054 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:44.049+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:44.114 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:44.109+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:44.213 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:44.208+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:44.340 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:44.335+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:44.359 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:44.354+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:44.766 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:44.760+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:44.812 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:44.808+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:45.044 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:45.038+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:45.090 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:45.085+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:45.184 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:45.178+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:45.355 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:45.351+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:45.377 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:45.371+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:45.803 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:45.798+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:45.866 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:45.861+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:46.045 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:46.037+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:46.130 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:46.126+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:46.154 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:46.148+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:46.336 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:46.330+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:46.349 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:46.344+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:46.801 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:46.796+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:46.820 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:46.815+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:47.008 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:47.002+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:47.093 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:47.088+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:47.203 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:47.197+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:47.327 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:47.322+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:47.395 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:47.391+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:47.796 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:47.791+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:47.810 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:47.805+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:47.981 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:47.972+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:48.134 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:48.129+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:48.187 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:48.182+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:48.333 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:48.325+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:48.423 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:48.419+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:48.770 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:48.765+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:48.848 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:48.844+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:48.998 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:48.991+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:49.149 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:49.143+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:49.188 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:49.182+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:49.313 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:49.307+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:49.409 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:49.404+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:49.791 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:49.786+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:49.886 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:49.882+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:50.037 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:50.032+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:50.104 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:50.098+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:50.150 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:50.145+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:50.311 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:50.307+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:50.406 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:50.401+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:50.814 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:50.809+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:50.928 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:50.923+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:50.995 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:50.989+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:51.151 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:51.146+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:51.183 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:51.179+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:51.349 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:51.337+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:51.389 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:51.379+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:51.860 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:51.855+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:51.882 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:51.878+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:52.000 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:51.995+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:52.164 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:52.159+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:52.236 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:52.231+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:52.307 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:52.302+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:52.436 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:52.427+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:52.861 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:52.856+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:52.867 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:52.863+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:52.985 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:52.980+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:53.199 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:53.194+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:53.280 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:53.276+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:53.359 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:53.354+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:53.484 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:53.480+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:53.893 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:53.888+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:53.901 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:53.897+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:54.035 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:54.029+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:54.185 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:54.177+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:54.275 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:54.270+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:54.378 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:54.366+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:54.455 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:54.451+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:54.929 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:54.925+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:54.946 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:54.941+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:55.036 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:55.030+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:55.220 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:55.215+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:55.339 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:55.335+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:55.407 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:55.402+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:55.423 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:55.419+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:55.910 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:55.905+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:55.968 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:55.964+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:56.043 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:56.038+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:56.218 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:56.212+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:56.364 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:56.358+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:56.379 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:56.374+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:56.441 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:56.437+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:56.897 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:56.892+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:56.965 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:56.961+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:57.002 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:56.997+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:57.206 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:57.201+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:57.359 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:57.356+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:57.405 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:57.399+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:57.484 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:57.479+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:57.914 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:57.908+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:58.017 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:58.012+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:58.049 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:58.044+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:58.163 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:58.158+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:58.354 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:58.350+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:58.370 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:58.365+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:58.460 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:58.456+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:58.884 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:58.879+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:58.989 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:58.984+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:43:59.069 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:43:59.064+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:43:59.167 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:43:59.161+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:43:59.331 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:43:59.328+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:43:59.400 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:43:59.395+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:43:59.496 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:43:59.491+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:43:59.853 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:43:59.847+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:43:59.960 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:43:59.955+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:00.088 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:00.084+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:00.197 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:00.192+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:00.322 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:00.317+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:00.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:00.388+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:00.525 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:00.520+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:00.898 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:00.894+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:00.924 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:00.921+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:01.050 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:01.046+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:01.156 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:01.150+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:01.351 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:01.345+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:01.445 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:01.440+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:01.485 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:01.481+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:01.893 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:01.889+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:01.953 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:01.948+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:02.061 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:02.057+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:02.164 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:02.160+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:02.378 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:02.373+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:02.444 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:02.439+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:02.495 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:02.490+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:02.931 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:02.927+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:03.004 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:03.000+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:03.112 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:03.107+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:03.188 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:03.185+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:03.408 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:03.404+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:03.467 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:03.463+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:03.507 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:03.502+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:03.701 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:44:03.696+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 41sec e221 v221) 2019-12-09T15:44:03.959 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:03.955+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:04.031 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:04.025+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:04.072 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:04.067+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:04.165 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:04.160+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:04.399 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:04.395+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:04.430 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:04.425+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:04.536 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:04.531+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:05.000 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:04.992+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:05.052 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:05.048+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:05.076 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:05.070+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:05.124 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:05.120+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:05.402 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:05.397+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:05.475 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:05.471+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:05.500 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:05.495+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:06.014 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:06.009+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:06.039 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:06.035+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:06.103 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:06.098+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:06.158 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:06.153+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:06.427 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:06.422+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:06.455 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:06.451+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:06.532 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:06.528+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:06.991 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:06.985+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:07.091 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:07.087+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:07.130 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:07.126+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:07.139 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:07.134+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:07.452 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:07.448+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:07.506 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:07.502+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:07.581 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:07.576+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:08.000 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:07.996+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:08.092 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:08.087+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:08.093 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:08.089+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:08.151 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:08.146+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:08.424 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:08.419+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:08.530 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:08.525+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:08.596 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:08.591+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:08.719 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:44:08.716+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 41sec e221 v221) 2019-12-09T15:44:08.969 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:08.964+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:09.084 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:09.079+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:09.135 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:09.131+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:09.206 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:09.201+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:09.391 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:09.386+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:09.571 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:09.566+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:09.619 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:09.614+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:10.014 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:10.010+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:10.128 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:10.124+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:10.145 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:10.140+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:10.185 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:10.181+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:10.366 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:10.361+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:10.612 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:10.608+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:10.673 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:10.668+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:11.048 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:11.045+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:11.121 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:11.116+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:11.180 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:11.175+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:11.229 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:11.224+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:11.377 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:11.372+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:11.585 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:11.581+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:11.662 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:11.657+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:12.019 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:12.014+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:12.137 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:12.132+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:12.182 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:12.175+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:12.275 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:12.266+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:12.347 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:12.342+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:12.602 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:12.598+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:12.699 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:12.694+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:13.048 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:44:13.040+0000 1694f700 -1 osd.3 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731814+0000 front 2019-12-09T15:42:50.931099+0000 (oldest deadline 2019-12-09T15:43:33.465865+0000) 2019-12-09T15:44:13.114 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:13.109+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:13.203 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:13.198+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:13.284 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:44:13.280+0000 1694f700 -1 osd.5 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.158751+0000 front 2019-12-09T15:42:49.111274+0000 (oldest deadline 2019-12-09T15:43:31.302663+0000) 2019-12-09T15:44:13.382 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:44:13.377+0000 1694f700 -1 osd.7 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.674075+0000 front 2019-12-09T15:42:48.424484+0000 (oldest deadline 2019-12-09T15:43:32.810802+0000) 2019-12-09T15:44:13.622 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:44:13.618+0000 1694f700 -1 osd.0 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.457234+0000 front 2019-12-09T15:42:49.484195+0000 (oldest deadline 2019-12-09T15:43:31.808030+0000) 2019-12-09T15:44:13.726 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:44:13.721+0000 1694f700 -1 osd.1 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.731600+0000 front 2019-12-09T15:42:49.718681+0000 (oldest deadline 2019-12-09T15:43:29.988578+0000) 2019-12-09T15:44:13.812 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:44:13.806+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 41sec e221 v221) 2019-12-09T15:44:14.159 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:44:14.155+0000 1694f700 -1 osd.4 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:49.676424+0000 front 2019-12-09T15:42:48.269298+0000 (oldest deadline 2019-12-09T15:43:29.687833+0000) 2019-12-09T15:44:14.250 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:44:14.245+0000 1694f700 -1 osd.2 221 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:42:52.150974+0000 front 2019-12-09T15:42:52.155809+0000 (oldest deadline 2019-12-09T15:43:35.145035+0000) 2019-12-09T15:44:25.423 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:32:01.688 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:25.433 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:32:01.698 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:25.445 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:32:01.667 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:25.450 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:32:01.672 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:25.452 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:32:01.674 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:25.459 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:32:01.681 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:25.696 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:32:01.961 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:25.724 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:32:01.946 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:25.729 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:32:01.952 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:45:12.028 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:12.024+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:13.004 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:12.999+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:14.018 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:14.003+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:15.040 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:15.036+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:16.059 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:16.056+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:17.018 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:17.013+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:18.038 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:18.034+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:18.677 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:18.673+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:19.010 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:19.007+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:19.741 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:19.726+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:20.061 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:20.057+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:20.722 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:20.715+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:21.069 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:21.065+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:21.740 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:21.720+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:22.048 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:22.044+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:22.774 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:22.759+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:23.078 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:23.073+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:23.785 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:23.768+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:24.107 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:24.103+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:24.813 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:24.796+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:25.102 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:25.098+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:25.808 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:25.791+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:26.101 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:26.097+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:26.783 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:26.766+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:27.087 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:27.083+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:27.771 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:27.755+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:28.066 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:28.062+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:28.809 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:28.792+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:29.087 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:29.083+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:29.821 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:29.802+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:30.110 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:30.107+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:30.844 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:30.825+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:31.142 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:31.139+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:31.833 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:31.816+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:32.110 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:32.106+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:32.880 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:32.858+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:33.117 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:33.114+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:33.921 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:33.902+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:34.076 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:34.073+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:34.938 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:34.919+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:35.054 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:35.050+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:35.961 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:35.942+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:36.026 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:36.023+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:37.022 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:37.003+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:37.029 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:37.025+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:37.985 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:37.981+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:38.052 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:38.040+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:38.989 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:38.985+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:39.128 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:39.114+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:39.992 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:39.988+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:40.200 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:40.178+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:41.012 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:41.008+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:41.188 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:41.172+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:41.973 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:41.969+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:42.234 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:42.217+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:42.999 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:42.996+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:43.259 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:43.243+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:43.985 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:43.981+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:44.284 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:44.267+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:44.952 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:44.948+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:45.296 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:45.277+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:45.991 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:45.987+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:46.298 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:46.281+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:47.031 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:47.027+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:47.301 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:47.282+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:48.024 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:48.019+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:48.355 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:48.336+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:48.985 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:48.979+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:49.344 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:49.329+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:49.967 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:49.963+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:50.349 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:50.331+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:50.947 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:50.942+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:51.342 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:51.325+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:51.918 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:51.914+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:52.392 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:52.375+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:52.920 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:52.916+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:53.408 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:53.393+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:53.972 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:53.968+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:54.399 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:54.383+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:55.011 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:55.005+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:55.389 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:55.366+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:56.047 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:56.042+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:56.427 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:56.409+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:57.026 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:57.021+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:57.449 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:57.432+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:57.883 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:45:57.876+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:45:58.032 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:58.025+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:45:58.034 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:58.028+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:58.466 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:58.449+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:58.835 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:45:58.828+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:45:58.919 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:45:58.914+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:45:59.059 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:59.054+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:45:59.064 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:45:59.059+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:45:59.089 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:45:59.082+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:45:59.485 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:59.468+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:45:59.492 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:45:59.481+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:45:59.828 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:45:59.821+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:45:59.891 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:45:59.884+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:00.041 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:00.035+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:00.042 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:00.038+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:00.060 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:00.051+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:00.392 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:00.386+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:00.446 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:00.439+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:00.480 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:00.463+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:00.773 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:00.767+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:00.802 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:00.796+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:00.848 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:00.843+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:01.054 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:01.048+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:01.055 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:01.051+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:01.088 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:01.080+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:01.369 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:01.363+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:01.525 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:01.502+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:01.538 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:01.521+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:01.764 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:01.757+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:01.805 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:01.798+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:01.893 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:01.886+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:02.089 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:02.082+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:02.091 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:02.085+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:02.128 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:02.121+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:02.330 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:02.324+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:02.560 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:02.538+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:02.572 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:02.556+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:02.754 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:02.748+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:02.799 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:02.792+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:02.870 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:02.865+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:03.078 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:03.072+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:03.082 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:03.078+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:03.118 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:03.111+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:03.302 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:03.296+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:03.533 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:03.511+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:03.551 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:03.529+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:03.789 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:03.778+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:03.801 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:03.793+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:03.887 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:03.883+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:04.112 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:04.106+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:04.114 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:04.109+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:04.120 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:04.113+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:04.351 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:04.346+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:04.551 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:04.526+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:04.565 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:04.546+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:04.774 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:04.767+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:04.841 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:04.835+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:04.929 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:04.925+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:05.124 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:05.115+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:05.146 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:05.140+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:05.147 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:05.143+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:05.308 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:05.303+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:05.524 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:05.517+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:05.558 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:05.540+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:05.782 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:05.776+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:05.839 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:05.832+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:05.925 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:05.921+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:06.141 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:06.134+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:06.195 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:06.188+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:06.206 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:06.196+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:06.289 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:06.275+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:06.574 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:06.556+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:06.587 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:06.570+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:06.756 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:06.749+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:06.867 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:06.861+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:06.912 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:06.906+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:07.156 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:07.149+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:07.203 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:07.194+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:07.204 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:07.200+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:07.308 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:07.303+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:07.560 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:07.534+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:07.576 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:07.556+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:07.790 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:07.783+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:07.883 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:07.877+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:07.904 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:07.899+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:08.162 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:08.158+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:08.166 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:08.161+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:08.186 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:08.180+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:08.345 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:08.339+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:08.580 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:08.563+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:08.591 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:08.576+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:08.824 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:08.817+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:08.848 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:08.841+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:08.895 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:08.890+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:09.153 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:09.145+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:09.192 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:09.187+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:09.203 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:09.191+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:09.347 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:09.343+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:09.569 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:09.552+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:09.578 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:09.565+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:09.801 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:09.795+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:09.879 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:09.873+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:09.929 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:09.924+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:10.170 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:10.165+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:10.172 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:10.168+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:10.176 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:10.171+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:10.332 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:10.327+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:10.621 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:10.605+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:10.629 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:10.617+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:10.777 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:10.769+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:10.929 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:10.924+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:10.935 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:10.928+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:11.137 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:11.132+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:11.138 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:11.134+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:11.187 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:11.179+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:11.316 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:11.312+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:11.609 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:11.586+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:11.617 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:11.605+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:11.809 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:11.801+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:11.898 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:11.891+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:11.931 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:11.926+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:12.179 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:12.173+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:12.181 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:12.176+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:12.189 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:12.182+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:12.361 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:12.351+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:12.595 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:12.576+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:12.613 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:12.591+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:12.865 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:12.851+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:12.909 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:12.902+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:12.961 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:12.952+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:13.215 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:13.210+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:13.220 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:13.216+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:13.227 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:13.219+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:13.322 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:13.317+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:13.563 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:13.557+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:13.596 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:13.578+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:13.861 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:13.853+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:13.947 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:13.941+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:13.989 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:13.985+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:14.243 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:14.238+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:14.245 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:14.241+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:14.281 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:14.275+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:14.294 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:14.289+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:14.607 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:14.585+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:14.619 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:14.603+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:14.857 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:14.851+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:14.956 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:14.950+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:15.012 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:15.006+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:15.211 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:15.204+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:15.217 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:15.213+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:15.267 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:15.260+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:15.294 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:15.290+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:15.579 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:15.556+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:15.593 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:15.575+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:15.840 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:15.833+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:15.977 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:15.970+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:15.981 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:15.975+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:16.222 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:16.218+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:16.225 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:16.222+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:16.286 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:16.280+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:16.291 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:16.286+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:16.598 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:16.592+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:16.632 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:16.614+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:16.821 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:16.814+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:16.975 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:16.968+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:16.986 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:16.980+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:17.195 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:17.188+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:17.197 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:17.192+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:17.268 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:17.263+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:17.276 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:17.269+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:17.647 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:17.624+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:17.661 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:17.643+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:17.791 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:17.784+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:17.971 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:17.965+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:17.999 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:17.992+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:18.217 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:18.211+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:18.223 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:18.217+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:18.232 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:18.226+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:18.321 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:18.312+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:18.664 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:18.641+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:18.678 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:18.660+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:18.758 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:18.752+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:18.965 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:18.956+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:18.974 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:18.966+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:19.241 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:19.230+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:19.267 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:19.261+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:19.268 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:19.264+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:19.306 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:19.300+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:19.692 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:19.670+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:19.707 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:19.688+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:19.802 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:19.795+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:19.989 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:19.982+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:20.021 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:20.015+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:20.200 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:20.193+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:20.239 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:20.228+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:20.240 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:20.232+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:20.316 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:20.308+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:20.679 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:20.673+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:20.711 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:20.694+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:20.818 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:20.809+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:20.963 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:20.953+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:21.015 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:21.011+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:21.180 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:21.173+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:21.190 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:21.185+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:21.191 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:21.187+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:21.362 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:21.357+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:21.761 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:21.738+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:21.776 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:21.757+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:21.848 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:21.841+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:21.955 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:21.949+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:22.058 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:22.054+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:22.174 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:22.165+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:22.176 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:22.171+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:22.185 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:22.177+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:22.340 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:22.332+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:22.769 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:22.745+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:22.783 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:22.764+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:22.902 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:22.896+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:22.963 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:22.956+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:23.079 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:23.074+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:23.181 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:23.174+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:23.189 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:23.182+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:23.191 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:23.186+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:23.357 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:23.352+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:23.799 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:23.770+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:23.814 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:23.789+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:23.907 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:23.900+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:24.015 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:24.008+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:24.085 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:24.081+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:24.164 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:24.156+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:24.165 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:24.161+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:24.193 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:24.186+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:24.362 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:24.355+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:24.776 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:24.752+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:24.785 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:24.772+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:24.937 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:24.931+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:24.972 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:24.964+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:25.137 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:25.132+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:25.173 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:25.168+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:25.208 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:25.200+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:25.210 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:25.204+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:25.389 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:25.385+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:25.765 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:25.744+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:25.782 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:25.777+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:25.936 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:25.930+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:25.940 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:25.933+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:26.098 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:26.093+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:26.139 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:26.133+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:26.195 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:26.188+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:26.197 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:26.191+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:26.352 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:26.347+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:26.761 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:26.738+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:26.779 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:26.757+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:26.906 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:26.900+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:26.958 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:26.952+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:27.135 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:27.130+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:27.146 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:27.139+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:27.197 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:27.188+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:27.202 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:27.191+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:27.347 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:27.341+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:27.747 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:27.726+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:27.761 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:27.740+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:27.881 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:27.874+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:27.927 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:27.920+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:28.107 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:28.101+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:28.117 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:28.111+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:28.214 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:28.207+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:28.216 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:28.211+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:28.304 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:28.298+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:28.731 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:28.708+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:28.744 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:28.727+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:28.880 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:28.872+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:28.909 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:28.904+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:29.135 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:29.128+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:29.142 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:29.137+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:29.224 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:29.220+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:29.226 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:29.223+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:29.323 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:29.317+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:29.458 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:46:29.451+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 40sec e227 v227) 2019-12-09T15:46:29.729 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:29.706+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:29.741 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:29.725+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:29.862 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:29.855+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:29.920 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:29.913+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:30.117 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:30.111+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:30.177 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:30.171+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:30.250 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:30.245+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:30.251 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:30.248+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:30.325 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:30.321+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:30.729 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:30.723+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:30.761 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:30.744+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:30.817 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:30.810+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:30.907 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:30.901+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:31.115 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:31.108+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:31.171 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:31.165+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:31.296 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:31.290+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:31.298 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:31.293+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:31.376 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:31.369+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:31.748 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:31.743+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:31.784 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:31.769+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:31.847 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:31.842+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:31.869 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:31.863+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:32.088 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:32.080+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:32.170 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:32.165+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:32.332 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:32.324+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:32.334 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:32.329+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:32.422 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:32.416+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:32.776 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:32.768+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:32.806 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:32.790+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:32.829 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:32.823+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:32.864 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:32.857+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:33.061 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:33.056+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:33.199 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:33.193+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:33.367 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:33.363+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:33.371 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:33.367+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:33.459 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:33.456+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:33.794 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:33.789+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:33.817 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:33.811+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:33.833 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:33.811+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:33.834 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:33.829+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:34.099 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:34.093+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:34.244 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:34.239+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:34.346 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:34.336+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:34.346 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:34.341+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:34.483 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:46:34.476+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 40sec e227 v227) 2019-12-09T15:46:34.494 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:34.488+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:34.768 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:34.764+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:34.785 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:34.776+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:34.810 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:34.805+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:34.852 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:34.846+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:35.105 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:35.100+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:35.262 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:35.257+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:35.382 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:35.375+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:35.384 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:35.378+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:35.522 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:35.517+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:35.815 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:35.811+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:35.825 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:35.820+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:35.840 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:35.830+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:35.842 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:35.829+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:36.143 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:36.137+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:36.314 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:36.308+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:36.354 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:36.346+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:36.356 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:36.351+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:36.495 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:36.490+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:36.805 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:36.783+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:36.817 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:36.802+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:36.854 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:36.848+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:36.875 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:36.870+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:37.118 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:37.113+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:37.302 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:46:37.297+0000 1694f700 -1 osd.2 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.347142+0000 front 2019-12-09T15:45:14.181025+0000 (oldest deadline 2019-12-09T15:45:56.978672+0000) 2019-12-09T15:46:37.359 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:37.352+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:37.359 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:37.356+0000 1694f700 -1 osd.0 227 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4355.0:114663 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5442695104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-09T15:46:37.472 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:46:37.466+0000 1694f700 -1 osd.3 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.429604+0000 front 2019-12-09T15:45:14.374753+0000 (oldest deadline 2019-12-09T15:45:59.437202+0000) 2019-12-09T15:46:37.869 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:37.859+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:37.878 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:37.873+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:37.891 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:46:37.879+0000 1694f700 -1 osd.7 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:15.435532+0000 front 2019-12-09T15:45:14.778599+0000 (oldest deadline 2019-12-09T15:46:00.495638+0000) 2019-12-09T15:46:37.918 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:37.898+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:38.082 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T15:46:38.075+0000 20e25700 -1 RGWWatcher::handle_error cookie 5442695104 err (107) Transport endpoint is not connected 2019-12-09T15:46:38.158 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:38.155+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:38.314 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:38.309+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:38.892 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:46:38.887+0000 1694f700 -1 osd.4 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.349499+0000 front 2019-12-09T15:45:14.135461+0000 (oldest deadline 2019-12-09T15:45:58.558115+0000) 2019-12-09T15:46:39.121 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:46:39.115+0000 1694f700 -1 osd.5 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.957928+0000 front 2019-12-09T15:45:15.699802+0000 (oldest deadline 2019-12-09T15:45:58.922396+0000) 2019-12-09T15:46:39.307 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:46:39.303+0000 1694f700 -1 osd.0 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:17.265628+0000 front 2019-12-09T15:45:17.265041+0000 (oldest deadline 2019-12-09T15:45:57.064116+0000) 2019-12-09T15:46:39.406 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:39.166+0000 1694f700 -1 osd.6 227 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:45:14.117239+0000 front 2019-12-09T15:45:14.107830+0000 (oldest deadline 2019-12-09T15:45:59.297540+0000) 2019-12-09T15:46:39.406 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:46:39.180+0000 1694f700 -1 osd.6 227 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=23 ondisk+retry+read+known_if_redirected e225) 2019-12-09T15:46:39.530 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:46:39.525+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 40sec e227 v227) 2019-12-09T15:46:49.756 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:34:26.021 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:49.801 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:34:26.066 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:49.860 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:34:26.083 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:49.861 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:34:26.083 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:49.862 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:34:26.084 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:49.863 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:34:26.086 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:49.968 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:34:26.233 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:50.012 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:34:26.234 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:50.044 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:34:26.266 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:47:33.761 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:33.755+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:33.939 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:33.934+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:33.942 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:33.937+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:34.728 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:34.722+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:34.912 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:34.907+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:34.925 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:34.920+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:34.963 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:34.957+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:35.683 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:35.677+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:35.887 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:35.884+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:35.916 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:35.912+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:36.015 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:36.010+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:36.675 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:36.668+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:36.891 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:36.885+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:36.891 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:36.885+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:36.960 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:36.957+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:36.999 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:36.992+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:37.004 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:36.998+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:37.691 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:37.685+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:37.860 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:37.855+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:37.880 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:37.876+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:37.986 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:37.980+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:38.014 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:38.009+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:38.036 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:38.029+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:38.719 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:38.713+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:38.823 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:38.818+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:38.916 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:38.911+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:38.996 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:38.990+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:39.049 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:39.043+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:39.058 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:39.054+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:39.755 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:39.749+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:39.873 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:39.868+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:39.944 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:39.940+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:40.025 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:40.017+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:40.025 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:40.021+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:40.046 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:40.040+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:40.451 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:40.447+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:40.711 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:40.706+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:40.880 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:40.876+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:40.900 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:40.894+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:41.013 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:41.008+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:41.064 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:41.058+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:41.064 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:41.059+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:41.456 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:41.450+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:41.689 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:41.684+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:41.871 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:41.866+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:41.883 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:41.879+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:42.033 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:42.028+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:42.046 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:42.041+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:42.080 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:42.075+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:42.460 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:42.455+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:42.734 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:42.729+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:42.891 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:42.886+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:42.927 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:42.923+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:43.066 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:43.060+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:43.275 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:43.075+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:43.276 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:43.067+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:43.476 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:43.471+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:43.703 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:43.698+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:43.917 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:43.911+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:43.929 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:43.924+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:44.029 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:44.023+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:44.056 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:44.049+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:44.125 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:44.120+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:44.511 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:44.506+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:44.752 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:44.746+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:44.935 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:44.927+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:44.958 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:44.954+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:45.009 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:45.004+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:45.047 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:45.037+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:45.145 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:45.140+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:45.488 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:45.484+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:45.758 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:45.753+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:45.886 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:45.882+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:45.924 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:45.919+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:45.987 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:45.981+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:46.077 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:46.070+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:46.127 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:46.123+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:46.520 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:46.515+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:46.796 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:46.790+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:46.889 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:46.884+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:46.953 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:46.947+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:46.965 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:46.958+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:47.090 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:47.085+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:47.097 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:47.091+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:47.574 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:47.568+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:47.769 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:47.764+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:47.847 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:47.843+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:47.977 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:47.972+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:48.006 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:48.002+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:48.066 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:48.062+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:48.118 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:48.113+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:48.555 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:48.551+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:48.783 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:48.778+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:48.846 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:48.841+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:48.952 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:48.947+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:49.030 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:49.025+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:49.080 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:49.076+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:49.160 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:49.156+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:49.605 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:49.599+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:49.754 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:49.749+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:49.890 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:49.884+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:49.976 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:49.972+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:50.036 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:50.033+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:50.092 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:50.087+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:50.157 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:50.153+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:50.572 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:50.568+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:50.756 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:50.752+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:50.854 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:50.849+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:50.976 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:50.972+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:51.018 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:51.013+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:51.121 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:51.113+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:51.196 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:51.193+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:51.560 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:51.555+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:51.740 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:51.735+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:51.816 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:51.810+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:51.969 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:51.963+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:51.975 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:51.971+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:52.093 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:52.088+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:52.187 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:52.182+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:52.537 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:52.530+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:52.751 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:52.747+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:52.850 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:52.846+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:52.963 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:52.957+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:52.984 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:52.980+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:53.079 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:53.075+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:53.219 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:53.214+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:53.531 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:53.526+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:53.786 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:53.782+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:53.840 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:53.835+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:53.982 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:53.977+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:53.982 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:53.976+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:54.131 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:54.127+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:54.187 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:54.182+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:54.564 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:54.559+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:54.813 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:54.809+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:54.858 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:54.854+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:54.990 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:54.985+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:54.996 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:54.991+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:55.154 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:55.150+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:55.173 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:55.169+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:55.615 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:55.610+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:55.817 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:55.812+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:55.901 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:55.889+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:55.981 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:55.975+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:56.010 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:56.002+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:56.190 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:56.186+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:56.202 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:56.198+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:56.575 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:56.571+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:56.855 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:56.850+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:56.913 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:56.908+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:57.006 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:57.002+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:57.015 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:57.011+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:57.173 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:57.168+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:57.231 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:57.228+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:57.588 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:57.584+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:57.901 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:57.897+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:57.950 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:57.944+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:58.006 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:58.002+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:58.025 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:58.020+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:58.223 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:58.218+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:58.263 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:58.258+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:58.578 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:58.574+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:58.957 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:58.952+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:58.990 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:58.985+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:59.013 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:47:59.008+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:47:59.030 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:59.025+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:47:59.223 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:47:59.219+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:47:59.232 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:47:59.228+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:47:59.625 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:47:59.621+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:47:59.930 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:47:59.926+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:47:59.954 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:47:59.949+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:47:59.985 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:47:59.979+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:00.036 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:00.032+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:00.220 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:00.215+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:00.267 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:00.262+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:00.615 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:00.611+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:00.965 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:00.960+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:00.998 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:00.993+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:01.010 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:01.005+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:01.011 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:01.006+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:01.189 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:01.184+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:01.239 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:01.235+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:01.625 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:01.620+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:01.965 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:01.960+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:01.983 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:01.980+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:02.024 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:02.020+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:02.060 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:02.055+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:02.187 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:02.182+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:02.264 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:02.259+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:02.582 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:02.577+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:02.999 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:02.995+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:03.025 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:03.020+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:03.032 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:03.027+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:03.078 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:03.073+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:03.194 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:03.189+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:03.237 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:03.233+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:03.619 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:03.614+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:04.014 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:04.009+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:04.032 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:04.028+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:04.076 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:04.070+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:04.098 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:04.093+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:04.189 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:04.183+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:04.230 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:04.225+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:04.593 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:04.588+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:04.994 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:04.989+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:05.030 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:05.026+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:05.038 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:05.032+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:05.093 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:05.088+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:05.195 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:05.190+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:05.238 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:05.233+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:05.625 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:05.621+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:06.037 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:06.030+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:06.043 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:06.038+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:06.071 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:06.065+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:06.089 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:06.084+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:06.172 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:06.161+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:06.227 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:06.222+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:06.608 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:06.603+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:06.994 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:06.989+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:07.047 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:07.041+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:07.070 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:07.066+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:07.133 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:07.128+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:07.205 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:07.201+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:07.218 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:07.213+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:07.624 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:07.619+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:07.947 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:07.943+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:08.085 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:08.080+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:08.117 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:08.113+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:08.154 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:08.143+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:08.175 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:08.170+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:08.252 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:08.248+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:08.589 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:08.585+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:08.931 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:08.925+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:09.047 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:09.041+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:09.076 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:09.072+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:09.122 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:09.116+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:09.166 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:09.161+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:09.297 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:09.292+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:09.620 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:09.615+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:09.901 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:09.895+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:10.017 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:48:10.011+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 43sec e233 v233) 2019-12-09T15:48:10.047 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:10.042+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:10.109 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:10.104+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:10.149 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:10.144+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:10.205 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:10.199+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:10.325 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:10.321+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:10.589 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:10.585+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:10.873 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:10.868+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:11.039 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:11.033+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:11.079 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:11.074+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:11.157 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:11.152+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:11.222 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:11.217+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:11.353 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:11.348+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:11.558 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:11.554+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:11.888 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:11.883+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:12.057 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:12.049+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:12.057 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:12.048+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:12.205 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:12.200+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:12.240 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:12.234+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:12.324 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:12.319+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:12.541 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:12.536+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:12.889 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:12.884+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:13.047 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:13.042+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:13.054 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:13.049+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:13.212 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:13.204+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:13.216 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:13.210+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:13.360 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:13.355+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:13.555 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:13.550+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:13.932 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:13.927+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:14.015 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:14.010+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:14.098 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:14.089+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:14.214 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:14.209+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:14.225 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:14.219+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:14.403 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:48:14.399+0000 1694f700 -1 osd.3 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:51.097793+0000 front 2019-12-09T15:46:51.078083+0000 (oldest deadline 2019-12-09T15:47:33.367254+0000) 2019-12-09T15:48:14.518 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:48:14.514+0000 1694f700 -1 osd.0 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:52.723121+0000 front 2019-12-09T15:46:52.709785+0000 (oldest deadline 2019-12-09T15:47:40.179346+0000) 2019-12-09T15:48:14.919 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:48:14.914+0000 1694f700 -1 osd.2 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.283550+0000 front 2019-12-09T15:46:49.284695+0000 (oldest deadline 2019-12-09T15:47:34.110647+0000) 2019-12-09T15:48:15.063 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:48:15.058+0000 1694f700 -1 osd.1 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:53.806617+0000 front 2019-12-09T15:46:53.148845+0000 (oldest deadline 2019-12-09T15:47:36.233497+0000) 2019-12-09T15:48:15.136 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:48:15.129+0000 1694f700 -1 osd.4 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:49.207052+0000 front 2019-12-09T15:46:49.210671+0000 (oldest deadline 2019-12-09T15:47:33.595088+0000) 2019-12-09T15:48:15.160 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:48:15.151+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 43sec e233 v233) 2019-12-09T15:48:15.181 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:48:15.176+0000 1694f700 -1 osd.7 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.101721+0000 front 2019-12-09T15:46:49.907901+0000 (oldest deadline 2019-12-09T15:47:33.345069+0000) 2019-12-09T15:48:15.267 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:48:15.262+0000 1694f700 -1 osd.5 233 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:46:50.933409+0000 front 2019-12-09T15:46:50.944111+0000 (oldest deadline 2019-12-09T15:47:36.099950+0000) 2019-12-09T15:48:20.562 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:35:56.826 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:20.573 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:35:56.838 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:20.592 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:35:56.815 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:20.594 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:35:56.816 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:20.600 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:35:56.822 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:20.605 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:35:56.827 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:20.700 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:35:56.965 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:20.722 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:35:56.944 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:20.743 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:35:56.965 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:50:02.931 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:02.926+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:03.430 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:03.426+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:03.937 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:03.931+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:04.099 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:04.092+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:04.220 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:04.216+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:04.431 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:04.425+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:04.893 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:04.880+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:04.925 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:04.919+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:05.142 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:05.136+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:05.228 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:05.224+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:05.481 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:05.476+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:05.921 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:05.915+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:05.931 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:05.926+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:06.109 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:06.103+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:06.221 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:06.216+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:06.419 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:06.414+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:06.438 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:06.433+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:06.447 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:06.442+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:06.924 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:06.918+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:06.929 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:06.924+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:07.115 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:07.109+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:07.233 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:07.229+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:07.414 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:07.408+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:07.429 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:07.424+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:07.467 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:07.463+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:07.893 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:07.888+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:07.893 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:07.887+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:08.084 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:08.078+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:08.285 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:08.278+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:08.392 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:08.388+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:08.418 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:08.414+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:08.506 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:08.501+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:08.880 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:08.875+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:08.882 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:08.877+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:09.128 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:09.122+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:09.287 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:09.283+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:09.415 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:09.411+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:09.423 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:09.420+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:09.519 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:09.509+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:09.872 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:09.866+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:09.876 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:09.870+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:10.129 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:10.124+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:10.284 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:10.279+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:10.399 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:10.393+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:10.460 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:10.455+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:10.517 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:10.512+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:10.878 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:10.872+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:10.889 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:10.884+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:11.106 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:11.099+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:11.265 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:11.260+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:11.413 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:11.408+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:11.415 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:11.411+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:11.559 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:11.555+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:11.863 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:11.858+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:11.877 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:11.872+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:12.152 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:12.142+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:12.299 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:12.294+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:12.375 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:12.370+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:12.382 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:12.377+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:12.529 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:12.520+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:12.857 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:12.850+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:12.919 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:12.913+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:13.157 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:13.150+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:13.278 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:13.274+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:13.367 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:13.363+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:13.415 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:13.411+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:13.558 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:13.553+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:13.859 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:13.854+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:13.884 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:13.877+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:14.129 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:14.122+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:14.234 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:14.230+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:14.419 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:14.415+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:14.446 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:14.441+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:14.592 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:14.587+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:14.863 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:14.857+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:14.898 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:14.892+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:15.102 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:15.097+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:15.282 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:15.277+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:15.415 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:15.410+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:15.429 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:15.424+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:15.620 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:15.616+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:15.848 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:15.842+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:15.901 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:15.896+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:16.152 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:16.146+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:16.236 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:16.231+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:16.411 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:16.406+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:16.468 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:16.462+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:16.602 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:16.597+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:16.833 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:16.827+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:16.948 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:16.942+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:17.140 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:17.132+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:17.234 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:17.229+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:17.437 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:17.432+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:17.522 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:17.517+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:17.596 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:17.591+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:17.791 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:17.785+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:17.924 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:17.919+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:18.154 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:18.149+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:18.209 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:18.204+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:18.428 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:18.423+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:18.487 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:18.482+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:18.628 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:18.624+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:18.787 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:18.781+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:18.967 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:18.961+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:19.196 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:19.191+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:19.199 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:19.194+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:19.393 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:19.387+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:19.487 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:19.482+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:19.613 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:19.608+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:19.817 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:19.812+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:19.982 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:19.977+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:20.215 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:20.210+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:20.216 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:20.211+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:20.438 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:20.432+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:20.457 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:20.451+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:20.617 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:20.611+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:20.808 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:20.798+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:20.982 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:20.976+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:21.188 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:21.183+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:21.233 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:21.229+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:21.428 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:21.423+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:21.428 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:21.424+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:21.645 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:21.641+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:21.857 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:21.852+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:22.028 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:22.024+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:22.187 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:22.183+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:22.220 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:22.214+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:22.421 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:22.418+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:22.422 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:22.418+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:22.680 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:22.676+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:22.854 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:22.849+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:23.065 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:23.062+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:23.211 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:23.206+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:23.213 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:23.208+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:23.419 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:23.415+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:23.439 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:23.435+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:23.689 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:23.684+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:23.881 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:23.876+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:24.091 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:24.083+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:24.186 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:24.181+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:24.230 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:24.226+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:24.395 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:24.391+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:24.445 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:24.441+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:24.705 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:24.700+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:24.933 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:24.928+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:25.128 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:25.124+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:25.210 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:25.206+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:25.254 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:25.250+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:25.440 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:25.436+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:25.448 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:25.445+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:25.731 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:25.726+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:25.915 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:25.911+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:26.133 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:26.126+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:26.228 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:26.223+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:26.275 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:26.270+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:26.404 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:26.399+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:26.417 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:26.413+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:26.712 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:26.707+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:26.970 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:26.963+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:27.109 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:27.105+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:27.267 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:27.262+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:27.278 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:27.274+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:27.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:27.391+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:27.405 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:27.401+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:27.759 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:27.754+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:27.949 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:27.945+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:28.104 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:28.099+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:28.286 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:28.281+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:28.325 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:28.319+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:28.362 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:28.358+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:28.389 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:28.385+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:28.764 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:28.759+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:28.935 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:28.930+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:29.519 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:29.281+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:29.519 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:29.404+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:29.519 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:29.434+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:29.519 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:29.109+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:29.520 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:29.247+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:29.780 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:29.775+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:29.918 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:29.914+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:30.101 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:30.097+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:30.231 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:30.228+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:30.302 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:30.297+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:30.459 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:30.454+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:30.469 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:30.464+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:30.823 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:30.819+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:30.966 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:30.962+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:31.153 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:31.147+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:31.253 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:31.249+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:31.308 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:31.301+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:31.482 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:31.478+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:31.505 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:31.500+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:31.794 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:31.790+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:32.012 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:32.007+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:32.168 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:32.162+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:32.217 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:32.213+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:32.287 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:32.283+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:32.521 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:32.516+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:32.549 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:32.544+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:32.828 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:32.823+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:32.977 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:32.972+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:33.187 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:33.182+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:33.215 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:33.211+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:33.314 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:33.310+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:33.513 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:33.508+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:33.523 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:33.518+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:33.884 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:33.879+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:34.018 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:34.013+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:34.227 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:34.223+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:34.228 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:34.221+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:34.300 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:34.295+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:34.508 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:34.504+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:34.512 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:34.507+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:34.901 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:34.897+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:35.010 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:35.005+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:35.250 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:35.244+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:35.263 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:35.258+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:35.352 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:35.347+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:35.499 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:35.495+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:35.559 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:35.554+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:35.668 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:50:35.663+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 44sec e239 v239) 2019-12-09T15:50:35.940 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:35.935+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:36.036 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:36.029+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:36.267 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:36.261+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:36.304 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:36.298+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:36.364 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:36.360+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:36.533 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:36.528+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:36.551 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:36.546+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:36.961 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:36.956+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:37.027 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:37.018+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:37.276 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:37.271+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:37.342 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:37.337+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:37.372 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:37.367+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:37.515 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:37.510+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:37.522 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:37.517+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:37.946 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:37.941+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:37.985 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:37.979+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:38.293 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:38.288+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:38.303 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:38.298+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:38.412 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:38.408+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:38.541 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:38.535+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:38.549 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:38.543+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:38.966 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:38.961+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:39.014 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:39.008+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:39.304 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:39.299+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:39.324 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:39.320+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:39.380 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:39.374+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:39.514 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:39.510+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:39.520 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:39.515+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:39.943 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:39.937+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:40.046 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:40.042+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:40.317 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:40.310+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:40.330 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:40.326+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:40.343 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:40.339+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:40.560 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:40.556+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:40.571 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:40.567+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:40.675 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:50:40.671+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 44sec e239 v239) 2019-12-09T15:50:40.953 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:40.949+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:41.067 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:41.062+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:41.321 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:41.315+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:41.360 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:41.351+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:41.376 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:41.370+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:41.584 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:41.578+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:41.614 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:41.606+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:41.984 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:41.980+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:42.044 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:42.039+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:42.357 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:42.351+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:42.357 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:42.352+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:42.358 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:42.354+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:42.620 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:42.615+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:42.640 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:42.635+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:43.005 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:43.000+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:43.071 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:43.065+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:43.312 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:43.306+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:43.320 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:43.315+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:43.321 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:43.315+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:43.581 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:43.576+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:43.688 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:43.683+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:44.018 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:44.012+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:44.058 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:44.052+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:44.280 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:44.274+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:44.306 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:44.300+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:44.362 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:44.356+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:44.572 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:44.566+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:44.653 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:44.647+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:45.006 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:45.000+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:50:45.023 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:50:45.017+0000 1694f700 -1 osd.7 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.126877+0000 front 2019-12-09T15:49:19.120458+0000 (oldest deadline 2019-12-09T15:50:02.485482+0000) 2019-12-09T15:50:45.244 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:50:45.238+0000 1694f700 -1 osd.1 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:19.654189+0000 front 2019-12-09T15:49:19.650352+0000 (oldest deadline 2019-12-09T15:50:04.149526+0000) 2019-12-09T15:50:45.266 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:50:45.261+0000 1694f700 -1 osd.5 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:18.384268+0000 front 2019-12-09T15:49:18.978789+0000 (oldest deadline 2019-12-09T15:50:03.901617+0000) 2019-12-09T15:50:45.334 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:50:45.328+0000 1694f700 -1 osd.4 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.900886+0000 front 2019-12-09T15:49:20.790030+0000 (oldest deadline 2019-12-09T15:50:04.274575+0000) 2019-12-09T15:50:45.559 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:50:45.554+0000 1694f700 -1 osd.2 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:21.826821+0000 front 2019-12-09T15:49:21.815105+0000 (oldest deadline 2019-12-09T15:50:05.862284+0000) 2019-12-09T15:50:45.691 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:50:45.686+0000 1694f700 -1 osd.0 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:22.180364+0000 front 2019-12-09T15:49:22.174213+0000 (oldest deadline 2019-12-09T15:50:05.626483+0000) 2019-12-09T15:50:45.773 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:50:45.767+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 44sec e239 v239) 2019-12-09T15:50:46.046 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:50:46.041+0000 1694f700 -1 osd.3 239 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:49:20.022968+0000 front 2019-12-09T15:49:20.123021+0000 (oldest deadline 2019-12-09T15:50:02.834411+0000) 2019-12-09T15:51:37.159 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:37.155+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:38.161 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:38.157+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:39.117 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:39.114+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:40.086 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:40.082+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:41.122 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:41.118+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:42.152 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:42.146+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:43.133 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:43.125+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:43.134 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:43.128+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:44.001 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:43.990+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:44.095 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:44.088+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:44.096 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:44.092+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:44.682 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:44.676+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:45.049 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:45.043+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:45.077 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:45.068+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:45.078 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:45.071+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:45.704 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:45.698+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:46.043 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:46.036+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:46.047 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:46.041+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:46.048 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:46.040+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:46.506 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:46.499+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:46.636 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:46.631+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:46.669 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:46.656+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:47.006 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:47.000+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:47.007 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:47.003+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:47.024 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:47.019+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:47.541 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:47.536+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:47.620 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:47.615+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:47.671 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:47.665+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:47.685 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:47.681+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:48.013 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:48.007+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:48.021 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:48.015+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:48.022 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:48.012+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:48.573 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:48.567+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:48.580 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:48.574+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:48.709 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:48.703+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:48.713 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:48.707+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:49.059 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:49.053+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:49.059 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:49.056+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:49.075 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:49.070+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:49.536 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:49.531+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:49.566 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:49.560+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:49.685 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:49.680+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:49.735 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:49.730+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:50.081 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:50.077+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:50.083 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:50.080+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:50.116 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:50.111+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:50.582 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:50.575+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:50.600 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:50.596+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:50.648 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:50.642+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:50.731 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:50.724+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:51.077 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:51.070+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:51.097 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:51.091+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:51.098 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:51.094+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:51.516 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:39:27.780 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:51:51.533 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:39:27.797 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:51:51.536 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:39:27.758 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:51:51.550 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:39:27.773 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:51:51.584 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:51.577+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:51.589 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:51.584+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:51.613 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:39:27.835 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:51:51.617 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:39:27.839 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:51:51.677 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:51.672+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:51.694 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:51.689+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:51.767 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:39:28.031 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:51:51.783 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:39:28.006 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:51:51.849 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:39:28.071 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:51:52.084 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:52.079+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:52.120 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:52.113+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:52.120 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:52.116+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:52.543 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:52.538+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:52.594 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:52.587+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:52.643 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:52.634+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:52.725 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:52.719+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:53.097 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:53.090+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:53.108 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:53.102+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:53.109 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:53.105+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:53.506 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:53.501+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:53.564 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:53.558+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:53.633 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:53.626+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:53.752 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:53.746+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:54.147 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:54.140+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:54.154 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:54.149+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:54.161 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:54.154+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:54.478 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:54.471+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:54.572 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:54.563+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:54.617 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:54.610+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:54.792 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:54.786+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:55.146 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:55.139+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:55.147 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:55.142+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:55.184 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:55.176+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:55.511 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:55.507+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:55.615 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:55.608+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:55.633 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:55.626+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:55.824 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:55.818+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:56.103 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:56.095+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:56.103 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:56.099+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:56.186 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:56.179+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:56.553 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:56.548+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:56.593 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:56.587+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:56.614 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:56.607+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:56.850 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:56.845+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:57.073 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:57.066+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:57.074 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:57.070+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:57.204 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:57.194+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:57.543 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:57.537+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:57.598 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:57.591+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:57.614 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:57.608+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:57.836 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:57.827+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:58.047 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:58.042+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:58.049 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:58.046+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:58.194 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:58.188+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:58.506 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:58.501+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:58.563 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:58.556+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:58.625 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:58.617+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:58.818 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:58.812+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:51:59.088 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:59.081+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:51:59.089 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:51:59.084+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:51:59.169 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:51:59.162+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:51:59.497 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:51:59.490+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:51:59.562 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:51:59.556+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:51:59.613 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:51:59.600+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:51:59.850 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:51:59.845+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:52:00.112 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:00.103+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:52:00.119 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:00.112+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:52:00.123 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:52:00.117+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:52:00.479 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:52:00.473+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:52:00.600 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:52:00.593+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:52:00.604 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:52:00.597+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:52:00.869 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:52:00.859+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:52:01.136 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:01.130+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:52:01.137 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:01.133+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:52:01.162 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:52:01.155+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:52:01.438 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:52:01.433+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:52:01.614 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:52:01.607+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:52:01.622 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:52:01.616+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:52:01.874 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:52:01.870+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:52:02.112 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:02.107+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:52:02.113 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:02.109+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:52:02.217 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:52:02.210+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:52:02.388 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:52:02.382+0000 1694f700 -1 osd.6 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e243) 2019-12-09T15:52:02.403 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:52:02.398+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:52:02.571 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:52:02.563+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:52:02.652 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:52:02.646+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:52:02.868 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:52:02.863+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:52:03.085 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:03.078+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:52:03.086 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:03.082+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:52:03.249 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:52:03.242+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:52:03.426 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:52:03.412+0000 1694f700 -1 osd.6 246 get_health_metrics reporting 127 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e243) 2019-12-09T15:52:03.452 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:52:03.448+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:52:03.547 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:52:03.541+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:52:03.649 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:52:03.642+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:52:03.898 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:52:03.892+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:52:04.116 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:04.106+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:52:04.117 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:04.109+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:52:04.274 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:52:04.268+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:52:04.466 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:52:04.461+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:52:04.491 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:52:04.476+0000 1694f700 -1 osd.6 246 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e243) 2019-12-09T15:52:04.516 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:52:04.505+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:52:04.680 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:52:04.671+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:52:04.941 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:52:04.935+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:52:05.149 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:05.143+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:52:05.150 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:05.146+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:52:05.267 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:52:05.261+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:52:05.444 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:52:05.437+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:52:05.515 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:52:05.508+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.132279+0000 front 2019-12-09T15:51:01.133147+0000 (oldest deadline 2019-12-09T15:51:45.661020+0000) 2019-12-09T15:52:05.517 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:52:05.499+0000 1694f700 -1 osd.6 246 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e243) 2019-12-09T15:52:05.719 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:52:05.712+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:52:05.963 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:52:05.957+0000 1694f700 -1 osd.2 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.882469+0000 front 2019-12-09T15:51:01.832096+0000 (oldest deadline 2019-12-09T15:51:45.913512+0000) 2019-12-09T15:52:06.174 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:06.164+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:50:57.746963+0000 front 2019-12-09T15:50:57.917855+0000 (oldest deadline 2019-12-09T15:51:42.976725+0000) 2019-12-09T15:52:06.178 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:06.169+0000 1694f700 -1 osd.0 246 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:52:06.306 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:52:06.300+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.688594+0000 front 2019-12-09T15:51:01.510052+0000 (oldest deadline 2019-12-09T15:51:43.763396+0000) 2019-12-09T15:52:06.445 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:52:06.438+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.515199+0000 front 2019-12-09T15:51:01.736247+0000 (oldest deadline 2019-12-09T15:51:43.775162+0000) 2019-12-09T15:52:06.624 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:52:06.610+0000 1694f700 -1 osd.6 247 get_health_metrics reporting 168 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e243) 2019-12-09T15:52:06.711 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:52:06.703+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:51:01.348070+0000 front 2019-12-09T15:51:01.621588+0000 (oldest deadline 2019-12-09T15:51:46.964793+0000) 2019-12-09T15:52:07.123 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:52:07.111+0000 3a280700 -1 osd.6 247 build_incremental_map_msg missing incremental map 244 2019-12-09T15:52:07.614 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:07.275+0000 1694f700 -1 osd.0 247 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:52:08.531 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:52:08.527+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:115261 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5413239904 gen 2] snapc 0=[] ondisk+write+known_if_redirected e241) 2019-12-09T15:52:09.537 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T15:52:09.472+0000 20e25700 -1 RGWWatcher::handle_error cookie 5413239904 err (107) Transport endpoint is not connected 2019-12-09T15:52:23.628 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:39:59.893 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:23.637 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:39:59.902 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:23.657 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:39:59.877 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:23.662 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:39:59.884 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:23.663 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:39:59.886 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:23.670 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:39:59.892 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:23.767 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:39:59.989 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:23.771 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:40:00.036 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:23.771 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:39:59.994 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:53:08.151 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:53:08.051+0000 1694f700 -1 osd.1 247 get_health_metrics reporting 1 slow ops, oldest is osd_repop(osd.0.0:29256 5.5 e242/231 5:a93a5511:::notify.2:head v 242'205) 2019-12-09T15:53:11.008 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:53:10.989+0000 3b282700 -1 osd.0 250 build_incremental_map_msg missing incremental map 250 2019-12-09T15:53:11.041 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:53:11.031+0000 38a7d700 -1 osd.5 250 build_incremental_map_msg missing incremental map 250 2019-12-09T15:53:11.084 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:53:11.072+0000 37a7b700 -1 osd.5 250 build_incremental_map_msg missing incremental map 250 2019-12-09T15:53:11.223 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:53:11.095+0000 3a280700 -1 osd.0 250 build_incremental_map_msg missing incremental map 250 2019-12-09T15:53:11.293 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:53:11.148+0000 37a7b700 -1 osd.0 250 build_incremental_map_msg missing incremental map 250 2019-12-09T15:53:11.594 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:53:11.521+0000 34a75700 -1 osd.4 250 build_incremental_map_msg missing incremental map 250 2019-12-09T15:53:11.596 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:53:11.533+0000 37a7b700 -1 osd.0 250 build_incremental_map_msg missing incremental map 250 2019-12-09T15:54:12.691 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:12.687+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:12.707 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:12.702+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:13.204 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:13.200+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:13.666 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:13.662+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:13.681 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:13.677+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:13.727 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:13.723+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:14.175 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:14.170+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:14.197 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:14.192+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:14.680 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:14.676+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:14.723 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:14.716+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:14.727 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:14.722+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:15.164 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:15.160+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:15.249 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:15.244+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:15.690 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:15.685+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:15.691 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:15.688+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:15.737 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:15.730+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:16.166 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:16.153+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:16.177 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:16.172+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:16.244 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:16.234+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:16.439 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:16.434+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:16.658 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:16.652+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:16.743 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:16.736+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:16.760 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:16.754+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:17.128 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:17.125+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:17.153 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:17.148+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:17.272 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:17.268+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:17.403 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:17.397+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:17.632 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:17.627+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:17.761 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:17.756+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:17.773 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:17.765+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:18.134 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:18.128+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:18.179 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:18.174+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:18.304 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:18.299+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:18.376 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:18.371+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:18.658 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:18.654+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:18.724 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:18.720+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:18.752 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:18.742+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:19.161 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:19.157+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:19.189 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:19.185+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:19.271 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:19.266+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:19.358 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:19.353+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:19.627 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:19.623+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:19.721 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:19.717+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:19.777 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:19.771+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:20.157 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:20.153+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:20.176 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:20.171+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:20.273 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:20.269+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:20.371 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:20.366+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:20.588 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:20.584+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:20.724 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:20.719+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:20.811 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:20.804+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:21.158 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:21.154+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:21.214 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:21.210+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:21.230 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:21.225+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:21.366 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:21.361+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:21.559 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:21.554+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:21.739 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:21.734+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:21.833 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:21.828+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:22.137 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:22.131+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:22.255 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:22.251+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:22.279 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:22.276+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:22.398 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:22.393+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:22.530 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:22.525+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:22.776 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:22.770+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:22.857 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:22.853+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:23.143 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:23.137+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:23.268 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:23.262+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:23.278 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:23.273+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:23.359 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:23.354+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:23.523 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:23.518+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:23.755 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:23.750+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:23.880 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:23.872+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:24.169 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:24.164+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:24.270 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:24.265+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:24.302 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:24.293+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:24.375 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:24.370+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:24.566 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:24.560+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:24.788 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:24.782+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:24.904 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:24.900+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:25.181 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:25.174+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:25.263 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:25.257+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:25.312 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:25.307+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:25.354 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:25.349+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:25.564 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:25.559+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:25.843 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:25.836+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:25.949 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:25.944+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:26.199 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:26.193+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:26.234 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:26.228+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:26.329 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:26.325+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:26.408 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:26.403+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:26.571 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:26.565+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:26.827 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:26.822+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:26.966 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:26.959+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:27.174 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:27.167+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:27.244 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:27.239+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:27.289 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:27.285+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:27.404 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:27.399+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:27.599 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:27.593+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:27.806 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:27.800+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:28.013 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:28.009+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:28.146 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:28.140+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:28.243 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:28.239+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:28.332 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:28.327+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:28.415 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:28.410+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:28.643 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:28.637+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:28.812 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:28.807+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:29.020 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:29.014+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:29.105 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:29.099+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:29.253 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:29.249+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:29.384 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:29.379+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:29.389 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:29.386+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:29.663 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:29.658+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:29.782 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:29.776+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:30.033 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:30.029+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:30.091 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:30.085+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:30.242 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:30.237+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:30.359 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:30.354+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:30.405 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:30.401+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:30.664 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:30.658+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:30.766 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:30.761+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:30.994 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:30.988+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:31.066 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:31.060+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:31.238 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:31.233+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:31.333 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:31.328+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:31.415 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:31.410+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:31.638 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:31.633+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:31.804 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:31.798+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:31.955 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:31.950+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:32.110 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:32.106+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:32.234 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:32.229+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:32.329 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:32.323+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:32.388 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:32.382+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:32.645 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:32.636+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:32.800 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:32.795+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:32.948 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:32.942+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:33.086 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:33.072+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:33.220 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:33.216+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:33.332 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:33.328+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:33.427 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:33.422+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:33.617 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:33.611+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:33.833 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:33.828+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:33.990 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:33.986+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:34.131 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:34.125+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:34.268 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:34.263+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:34.312 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:34.305+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:34.405 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:34.401+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:34.639 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:34.634+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:34.800 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:34.794+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:34.963 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:34.958+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:35.084 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:35.079+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:35.297 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:35.293+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:35.338 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:35.333+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:35.408 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:35.403+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:35.606 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:35.601+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:35.756 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:35.750+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:35.926 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:35.922+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:36.051 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:36.045+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:36.258 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:36.254+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:36.352 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:36.344+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:36.421 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:36.415+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:36.582 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:36.576+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:36.772 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:36.766+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:36.933 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:36.929+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:37.010 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:37.004+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:37.268 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:37.263+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:37.370 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:37.365+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:37.421 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:37.415+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:37.568 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:37.563+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:37.794 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:37.789+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:37.902 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:37.897+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:37.996 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:37.990+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:38.232 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:38.224+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:38.381 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:38.376+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:38.405 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:38.400+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:38.538 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:38.533+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:38.840 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:38.834+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:38.946 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:38.941+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:39.005 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:38.999+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:39.275 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:39.270+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:39.405 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:39.401+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:39.432 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:39.424+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:39.586 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:39.581+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:39.802 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:39.796+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:39.927 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:39.923+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:40.025 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:40.020+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:40.299 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:40.295+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:40.406 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:40.402+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:40.460 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:40.454+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:40.622 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:40.617+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:40.819 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:40.814+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:40.898 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:40.893+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:41.054 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:41.049+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:41.297 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:41.293+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:41.418 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:41.413+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:41.485 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:41.481+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:41.581 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:41.571+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:41.785 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:41.780+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:41.895 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:41.891+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:42.078 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:42.072+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:42.283 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:42.279+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:42.436 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:42.430+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:42.461 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:42.456+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:42.576 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:42.570+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:42.769 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:42.763+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:42.914 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:42.909+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:43.040 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:43.035+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:43.266 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:43.261+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:43.423 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:43.418+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:43.442 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:43.437+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:43.542 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:43.532+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:43.742 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:43.736+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:43.952 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:43.947+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:44.024 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:44.018+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:44.294 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:44.289+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:44.396 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:44.391+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:44.437 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:44.431+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:44.556 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:44.551+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:44.755 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:44.749+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:44.925 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:44.920+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:45.034 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:45.028+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:45.336 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:45.331+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:45.388 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:45.379+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:45.460 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:45.453+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:45.592 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:45.587+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:45.770 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:45.763+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:45.969 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:45.963+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:46.000 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:45.993+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:46.315 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:46.309+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:46.401 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:46.391+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:46.495 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:46.490+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:46.572 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:46.566+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:46.820 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:46.814+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:46.967 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:46.962+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:46.982 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:46.976+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:47.083 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:54:47.077+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 42sec e252 v252) 2019-12-09T15:54:47.313 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:47.307+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:47.430 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:47.424+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:47.457 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:47.447+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:47.601 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:47.595+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:47.872 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:47.865+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:47.938 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:47.933+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:47.966 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:47.954+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:48.266 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:48.262+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:48.408 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:48.403+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:48.440 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:48.436+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:48.559 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:48.555+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:48.827 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:48.820+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:48.921 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:48.916+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:48.929 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:48.924+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:49.227 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:49.222+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:49.367 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:49.362+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:49.436 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:49.429+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:49.581 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:49.575+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:49.803 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:49.797+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:49.892 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:49.887+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:49.901 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:49.895+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:50.196 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:50.191+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:50.369 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:50.364+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:50.427 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:50.422+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:50.570 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:50.557+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:50.792 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:50.787+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:50.906 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:50.900+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:50.911 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:50.907+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:51.251 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:51.246+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:51.412 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:51.407+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:51.462 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:51.457+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:51.550 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:51.543+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:51.809 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:51.803+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:51.900 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:51.896+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:51.924 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:51.919+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:52.098 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:54:52.093+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 42sec e252 v252) 2019-12-09T15:54:52.276 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:52.272+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:52.369 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:52.364+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:52.440 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:52.435+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:52.591 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:52.585+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:52.827 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:52.821+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:52.856 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:52.851+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:52.955 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:52.949+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:53.247 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:53.242+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:53.404 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:53.398+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:53.471 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:53.467+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:53.580 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:53.575+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:53.814 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:53.809+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:53.884 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:53.880+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:53.947 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:53.941+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:54.209 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:54.203+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:54.407 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:54.403+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:54.498 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:54.492+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:54.612 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:54.607+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:54.820 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:54.815+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:54.894 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:54.889+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:54.934 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:54.928+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:55.172 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:55.166+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:55.405 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:55.401+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:55.474 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:55.469+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:55.662 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:55.656+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:55.871 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:55.865+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:55.884 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:55.880+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:55.937 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:55.931+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:56.177 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:56.173+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:56.450 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:54:56.446+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.717641+0000 front 2019-12-09T15:53:32.887935+0000 (oldest deadline 2019-12-09T15:54:16.015083+0000) 2019-12-09T15:54:56.461 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:54:56.457+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:30.829644+0000 front 2019-12-09T15:53:32.790239+0000 (oldest deadline 2019-12-09T15:54:14.030232+0000) 2019-12-09T15:54:56.620 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:54:56.614+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:33.477554+0000 front 2019-12-09T15:53:30.765759+0000 (oldest deadline 2019-12-09T15:54:13.335680+0000) 2019-12-09T15:54:56.894 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:54:56.888+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.784685+0000 front 2019-12-09T15:53:33.251910+0000 (oldest deadline 2019-12-09T15:54:12.198985+0000) 2019-12-09T15:54:56.898 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:54:56.893+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:27.680820+0000 front 2019-12-09T15:53:32.412252+0000 (oldest deadline 2019-12-09T15:54:11.973206+0000) 2019-12-09T15:54:56.969 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:54:56.963+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:28.113831+0000 front 2019-12-09T15:53:28.115585+0000 (oldest deadline 2019-12-09T15:54:12.512786+0000) 2019-12-09T15:54:57.150 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:54:57.146+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:53:32.906936+0000 front 2019-12-09T15:53:32.905466+0000 (oldest deadline 2019-12-09T15:54:15.169005+0000) 2019-12-09T15:54:57.176 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:54:57.166+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 42sec e252 v252) 2019-12-09T15:55:12.467 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:42:48.731 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:55:12.477 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:42:48.742 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:55:12.520 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:42:48.741 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:55:12.521 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:42:48.743 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:55:12.522 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:42:48.745 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:55:12.536 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:42:48.758 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:55:12.615 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:42:48.880 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:55:12.688 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:42:48.910 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:55:12.690 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:42:48.912 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:55:37.167 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:55:37.156+0000 37a7b700 -1 osd.6 257 build_incremental_map_msg missing incremental map 257 2019-12-09T15:55:37.414 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:55:37.382+0000 3827c700 -1 osd.6 257 build_incremental_map_msg missing incremental map 257 2019-12-09T15:55:37.426 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:55:37.402+0000 3a280700 -1 osd.6 257 build_incremental_map_msg missing incremental map 257 2019-12-09T15:55:37.522 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:55:37.479+0000 38a7d700 -1 osd.6 257 build_incremental_map_msg missing incremental map 257 2019-12-09T15:55:37.806 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:55:37.674+0000 3aa81700 -1 osd.6 257 build_incremental_map_msg missing incremental map 257 2019-12-09T15:55:56.088 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:55:56.081+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:55:57.059 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:55:57.052+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:55:57.575 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:55:57.571+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:55:58.014 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:55:58.007+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:55:58.370 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:55:58.361+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:55:58.425 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:55:58.420+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:55:58.542 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:55:58.537+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:55:58.998 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:55:58.993+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:55:59.034 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:55:59.028+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:55:59.381 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:55:59.376+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:55:59.418 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:55:59.410+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:55:59.551 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:55:59.547+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:55:59.616 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:55:59.600+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:00.042 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:00.037+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:00.078 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:00.072+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:00.421 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:00.415+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:00.448 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:00.440+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:00.546 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:00.540+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:00.547 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:00.543+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:00.609 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:00.603+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:01.079 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:01.074+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:01.111 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:01.100+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:01.391 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:01.387+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:01.434 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:01.427+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:01.539 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:01.535+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:01.540 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:01.537+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:01.645 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:01.637+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:02.081 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:02.076+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:02.129 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:02.125+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:02.355 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:02.349+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:02.428 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:02.419+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:02.524 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:02.516+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:02.525 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:02.520+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:02.666 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:02.660+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:03.060 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:03.054+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:03.115 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:03.110+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:03.334 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:03.328+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:03.447 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:03.441+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:03.547 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:03.541+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:03.552 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:03.544+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:03.640 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:03.633+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:04.110 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:04.104+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:04.112 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:04.104+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:04.380 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:04.374+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:04.487 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:04.480+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:04.513 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:04.507+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:04.514 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:04.510+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:04.669 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:04.662+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:05.074 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:05.068+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:05.080 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:05.074+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:05.423 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:05.418+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:05.501 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:05.494+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:05.513 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:05.506+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:05.515 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:05.510+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:05.656 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:05.649+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:06.066 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:06.059+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:06.110 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:06.102+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:06.458 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:06.452+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:06.473 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:06.464+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:06.515 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:06.509+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:06.518 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:06.515+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:06.702 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:06.695+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:07.032 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:07.027+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:07.147 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:07.140+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:07.456 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:07.451+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:07.498 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:07.491+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:07.499 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:07.495+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:07.503 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:07.496+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:07.734 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:07.725+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:08.074 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:08.065+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:08.162 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:08.155+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:08.425 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:08.420+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:08.483 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:08.478+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:08.484 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:08.481+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:08.501 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:08.494+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:08.770 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:08.756+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:08.940 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:08.934+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:09.107 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:09.103+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:09.139 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:09.133+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:09.454 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:09.448+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:09.461 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:09.456+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:09.510 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:09.505+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:09.511 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:09.507+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:09.825 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:09.820+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:09.900 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:09.896+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:10.117 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:10.112+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:10.153 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:10.146+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:10.474 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:10.467+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:10.504 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:10.498+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:10.532 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:10.526+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:10.543 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:10.532+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:10.860 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:10.853+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:10.880 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:10.875+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:11.131 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:11.123+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:11.152 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:11.147+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:11.458 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:11.452+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:11.552 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:11.547+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:11.555 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:11.550+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:11.557 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:11.554+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:11.910 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:11.904+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:11.947 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:11.939+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 62 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:12.134 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:12.128+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:12.144 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:12.137+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:12.483 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:12.476+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:12.523 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:12.518+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:12.526 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:12.522+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:12.535 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:12.528+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:12.961 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:12.943+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 126 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:12.964 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:12.958+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:13.095 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:13.090+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:13.194 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:13.188+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:13.490 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:13.483+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:13.500 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:13.495+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:13.555 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:13.549+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:13.558 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:13.555+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:13.965 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:13.958+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:13.985 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:13.979+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:14.024 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:14.008+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:14.089 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:14.083+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:14.207 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:14.201+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:14.476 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:14.471+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:14.528 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:14.521+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:14.528 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:14.524+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:14.530 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:14.524+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:14.992 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:14.985+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:15.021 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:14.999+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:15.035 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:15.017+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:15.045 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:15.039+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:15.262 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:15.255+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:15.519 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:15.514+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:15.527 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:15.518+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:15.527 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:15.521+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:15.549 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:15.542+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:15.982 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:15.975+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:16.025 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:16.005+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:16.040 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:16.021+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:16.086 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:16.080+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:16.271 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:16.262+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:16.512 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:16.506+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:16.519 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:16.509+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:16.539 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:16.529+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:16.550 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:16.544+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:16.982 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:16.975+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:17.058 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:17.034+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:17.073 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:17.053+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:17.125 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:17.120+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:17.313 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:17.307+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:17.484 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:17.477+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:17.484 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:17.481+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:17.538 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:17.532+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:17.581 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:17.576+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:18.005 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:17.998+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:18.091 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:18.086+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:18.100 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:18.091+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:18.130 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:18.112+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:18.296 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:18.290+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:18.525 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:18.514+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:18.525 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:18.517+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:18.533 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:18.528+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:18.610 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:18.602+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:19.058 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:19.051+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:19.073 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:19.067+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:19.121 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:19.098+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:19.134 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:19.116+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:19.300 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:19.294+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:19.498 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:19.491+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:19.500 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:19.495+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:19.549 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:19.543+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:19.582 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:19.576+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:20.041 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:20.034+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:20.101 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:20.094+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:20.190 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:20.165+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:20.204 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:20.185+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:20.274 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:20.268+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:20.512 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:20.508+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:20.534 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:20.530+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:20.537 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:20.533+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:20.589 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:20.582+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:21.040 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:21.034+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:21.136 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:21.129+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:21.225 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:21.201+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:21.238 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:21.221+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:21.302 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:21.295+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:21.524 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:21.518+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:21.580 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:21.573+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:21.581 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:21.575+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:21.611 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:21.603+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:22.031 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:22.024+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:22.174 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:22.167+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:22.263 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:22.257+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:22.290 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:22.273+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:22.291 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:22.283+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:22.532 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:22.525+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:22.543 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:22.532+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:22.544 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:22.535+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:22.570 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:22.563+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:23.007 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:23.001+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:23.177 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:23.171+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:23.269 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:23.247+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:23.283 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:23.265+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:23.306 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:23.299+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:23.543 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:23.536+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:23.545 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:23.539+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:23.572 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:23.564+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:23.577 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:23.570+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:24.034 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:24.028+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:24.156 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:24.150+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:24.290 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:24.283+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:24.308 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:24.302+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:24.342 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:24.322+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:24.512 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:24.499+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:24.512 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:24.503+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:24.557 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:24.553+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:24.602 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:24.593+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:24.995 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:24.987+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:25.185 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:25.176+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:25.291 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:25.285+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:25.353 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:25.348+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:25.373 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:25.360+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:25.527 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:25.524+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:25.529 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:25.526+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:25.582 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:25.578+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:25.667 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:25.605+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:26.028 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:26.023+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:26.223 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:26.216+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:26.264 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:26.258+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:26.362 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:26.343+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:26.374 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:26.358+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:26.491 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:26.485+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:26.493 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:26.488+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:26.589 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:26.572+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:26.597 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:26.592+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:27.008 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:27.004+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:27.225 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:27.218+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:27.270 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:27.263+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:27.364 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:27.335+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:27.375 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:27.353+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:27.542 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:27.538+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:27.544 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:27.540+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:27.558 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:27.553+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:27.621 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:27.612+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:27.839 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:56:27.830+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e258 v258) 2019-12-09T15:56:28.002 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:27.997+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:28.098 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:44:04.362 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:56:28.127 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:44:04.349 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:56:28.130 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:44:04.352 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:56:28.131 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:44:04.395 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:56:28.133 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:44:04.355 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:56:28.137 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:44:04.358 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:56:28.246 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:28.240+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:28.293 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:44:04.512 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T15:56:28.302 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:44:04.567 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T15:56:28.309 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:44:04.531 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T15:56:28.314 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:28.308+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:28.333 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:28.324+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:28.363 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:28.346+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:28.524 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:28.519+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:28.525 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:28.521+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:28.572 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:28.564+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:28.644 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:28.639+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:28.977 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:28.973+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:29.255 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:29.248+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:29.349 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:29.343+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:29.350 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:29.344+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:29.378 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:29.363+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:29.516 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:29.512+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:29.517 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:29.514+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:29.548 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:29.543+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:29.682 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:29.675+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:29.975 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:29.970+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:30.254 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:30.246+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:30.363 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:30.357+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:30.394 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:30.388+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:30.424 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:30.409+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:30.487 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:30.482+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:30.488 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:30.484+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:30.562 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:30.557+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:30.698 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:30.691+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:31.022 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:31.016+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:31.235 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:31.229+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:31.406 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:31.400+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:31.415 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:31.382+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:31.430 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:31.411+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:31.502 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:31.496+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:31.506 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:31.502+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:31.565 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:31.559+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:31.718 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:31.711+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:31.993 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:31.986+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:32.231 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:32.224+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:32.379 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:32.372+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:32.406 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:32.387+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:32.419 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:32.402+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:32.509 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:32.505+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:32.513 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:32.506+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:32.564 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:32.559+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:32.688 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:32.680+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:32.873 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:56:32.862+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e258 v258) 2019-12-09T15:56:33.011 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:33.005+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:33.215 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:33.208+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:33.396 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:33.388+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:33.415 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:33.408+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:33.432 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:33.412+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:33.506 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:33.498+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:33.508 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:33.503+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:33.535 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:33.530+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:33.675 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:33.666+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:33.976 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:33.971+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:34.261 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:34.253+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:34.442 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:34.435+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:34.486 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:34.461+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:34.495 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:34.481+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:34.506 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:34.501+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:34.540 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:34.534+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:34.540 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:34.536+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:34.729 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:34.721+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:34.933 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:34.927+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:35.262 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:35.255+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:35.466 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:35.442+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:35.478 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:35.472+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:35.480 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:35.461+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:35.558 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:35.551+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:35.592 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:35.585+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:35.596 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:35.588+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:35.691 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:35.684+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:35.941 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:35.937+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:36.285 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:36.278+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:36.439 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:36.433+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:36.471 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:36.453+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:36.484 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:36.478+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:36.583 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:36.578+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:36.587 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:36.583+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:36.593 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:36.588+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:36.736 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:36.728+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:36.961 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:36.956+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:37.332 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:37.324+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:37.454 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:37.432+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:37.468 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:37.450+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:37.471 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:37.464+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:37.596 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:37.591+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:37.605 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:37.599+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:37.606 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:37.602+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:37.749 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:37.742+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:37.919 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:56:37.884+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e258 v258) 2019-12-09T15:56:37.929 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:37.925+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:38.337 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:38.330+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:38.438 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:38.430+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:38.442 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:38.435+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:38.473 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:38.455+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:38.618 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:38.613+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:38.622 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:38.617+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:38.623 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:38.618+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:38.795 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:38.784+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:38.902 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:38.898+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:39.355 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:39.349+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:39.469 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:39.464+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:39.486 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:39.480+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:39.503 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:39.487+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:39.609 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:39.596+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:39.609 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:39.598+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:39.668 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:39.663+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:39.806 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:39.800+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:39.883 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:39.879+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:40.396 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:40.390+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:40.475 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:40.456+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:40.490 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:40.472+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:40.514 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:40.508+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:40.558 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:40.552+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:40.567 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:40.558+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:40.637 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:40.623+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:40.838 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:40.832+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:40.924 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:40.917+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:41.424 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:41.418+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:41.460 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:41.439+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:41.475 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:41.456+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:41.525 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:41.519+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:41.547 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:41.539+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:41.548 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:41.543+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:41.638 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:41.633+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:41.798 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:41.792+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:41.958 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:41.953+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:42.473 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:42.467+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:42.481 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:42.457+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:42.499 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:42.476+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:42.516 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:42.509+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:42.525 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:42.518+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:42.526 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:42.512+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:42.667 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:42.662+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:42.856 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:42.850+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:42.920 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:56:42.914+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e258 v258) 2019-12-09T15:56:42.938 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:42.933+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:43.499 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:43.495+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:43.519 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:43.498+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:43.528 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:43.522+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:43.529 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:43.515+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:43.538 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:43.529+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:43.570 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:43.564+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:43.630 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:43.623+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:43.877 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:43.871+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:43.939 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:43.934+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:44.490 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:44.468+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:44.506 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:44.487+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:44.515 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:44.504+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:44.532 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:44.524+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:44.534 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:44.528+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:44.581 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:44.575+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:44.624 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:44.617+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:44.874 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:44.868+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:44.948 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:44.943+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:45.473 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:45.468+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:45.476 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:45.469+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:45.495 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:45.483+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:45.544 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:45.540+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:45.547 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:45.543+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:45.577 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:45.572+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:45.631 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:45.626+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:45.912 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:45.907+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:45.987 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:45.984+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:46.493 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:46.486+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:46.533 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:46.528+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:46.535 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:46.531+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:46.536 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:46.532+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:46.561 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:46.548+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:46.593 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:46.588+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:46.637 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:46.632+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:46.917 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:46.911+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:46.970 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:46.964+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:47.494 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:47.489+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:47.497 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:47.493+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:47.545 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:47.539+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:47.598 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:47.576+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:47.610 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:47.593+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:47.623 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:47.617+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:47.629 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:47.624+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:47.875 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:47.869+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:47.925 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:47.919+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:47.943 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:56:47.936+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e258 v258) 2019-12-09T15:56:48.520 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:48.508+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:48.520 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:48.511+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:48.555 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:48.549+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:48.628 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:48.623+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:48.634 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:48.628+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:48.660 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:48.645+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:48.666 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:48.661+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:48.880 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:48.873+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:48.885 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:48.877+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:49.541 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:49.530+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:49.553 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:49.547+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:49.558 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:49.553+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:49.673 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:49.668+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:49.686 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:49.679+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:49.687 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:49.681+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:49.728 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:49.711+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:49.844 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:49.839+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:49.918 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:49.913+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:50.567 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:50.561+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:50.596 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:50.589+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:50.597 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:50.592+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:50.673 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:50.668+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:50.675 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:50.669+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:50.767 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:50.762+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:50.797 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:50.783+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:50.816 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:50.809+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:50.957 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:50.951+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:51.600 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:51.593+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:51.601 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:51.597+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:51.619 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:51.613+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:51.668 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:51.662+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:51.704 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:51.699+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:51.824 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:51.819+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:51.847 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:51.834+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:51.851 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:51.845+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:51.920 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:51.913+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:52.615 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:52.608+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:52.616 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:52.611+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:52.625 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:52.620+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:52.668 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:52.663+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:52.715 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:52.710+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:52.844 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:52.819+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:52.860 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:52.840+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:52.871 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:52.866+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:52.900 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:52.894+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:52.966 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:56:52.957+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e258 v258) 2019-12-09T15:56:53.587 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:53.582+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:53.623 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:53.615+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:53.625 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:53.621+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:53.634 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:53.629+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:53.689 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:53.684+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:53.854 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:53.829+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:53.866 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:53.848+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:53.871 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:53.865+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:53.874 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:53.867+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:54.580 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:54.573+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:54.643 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:54.634+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:54.680 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:54.673+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:54.680 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:54.677+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:54.683 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:54.677+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:54.857 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:54.835+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:54.872 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:54.852+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:54.880 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:54.873+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:54.889 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:54.884+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:55.633 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:55.626+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:55.644 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:55.638+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:55.704 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:55.699+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:55.732 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:55.708+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:55.732 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:55.710+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:55.908 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:55.902+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:55.910 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:55.905+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:55.912 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:55.890+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:55.926 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:55.908+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:56.665 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:56.658+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:56.689 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:56.680+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:56.692 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:56.685+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:56.714 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:56.707+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:56.716 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:56.710+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:56.879 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:56.875+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:56.886 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:56.879+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:56.974 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:56.951+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:56.989 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:56.969+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:57.644 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:56:57.638+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:11.948473+0000 front 2019-12-09T15:55:12.385286+0000 (oldest deadline 2019-12-09T15:55:57.830499+0000) 2019-12-09T15:56:57.682 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:56:57.675+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044009+0000 front 2019-12-09T15:55:13.852691+0000 (oldest deadline 2019-12-09T15:55:59.314377+0000) 2019-12-09T15:56:57.703 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:56:57.696+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:10.460623+0000 front 2019-12-09T15:55:10.461176+0000 (oldest deadline 2019-12-09T15:55:55.113572+0000) 2019-12-09T15:56:57.735 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:57.730+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.771154+0000 front 2019-12-09T15:55:14.542596+0000 (oldest deadline 2019-12-09T15:56:00.426135+0000) 2019-12-09T15:56:57.737 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:57.732+0000 1694f700 -1 osd.0 258 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:56:57.869 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:56:57.864+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:12.532256+0000 front 2019-12-09T15:55:12.593491+0000 (oldest deadline 2019-12-09T15:55:58.414987+0000) 2019-12-09T15:56:57.912 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:56:57.906+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:55:14.044171+0000 front 2019-12-09T15:55:13.990752+0000 (oldest deadline 2019-12-09T15:55:57.444306+0000) 2019-12-09T15:56:57.967 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:57.961+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:55:33.225080+0000 (oldest deadline 2019-12-09T15:56:13.225080+0000) 2019-12-09T15:56:57.999 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T15:56:57.982+0000 1694f700 -1 osd.6 258 get_health_metrics reporting 167 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:56:58.032 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:56:58.023+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e258 v258) 2019-12-09T15:56:59.006 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:56:58.952+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:57:00.244 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:57:00.226+0000 1694f700 -1 osd.0 260 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:115671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5040021616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-09T15:57:01.088 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T15:57:00.847+0000 20e25700 -1 RGWWatcher::handle_error cookie 5040021616 err (107) Transport endpoint is not connected 2019-12-09T15:58:20.291 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:20.285+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:21.310 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:21.306+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:21.383 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:21.378+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:21.585 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:21.580+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:22.028 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:22.022+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:22.289 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:22.284+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:22.397 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:22.392+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:22.549 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:22.537+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:22.769 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:22.765+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:22.995 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:22.990+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:23.246 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:23.241+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:23.387 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:23.382+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:23.408 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:23.403+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:23.504 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:23.499+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:23.745 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:23.740+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:23.964 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:23.960+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:24.247 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:24.242+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:24.419 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:24.412+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:24.451 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:24.446+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:24.488 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:24.483+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:24.722 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:24.718+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:24.939 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:24.934+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:25.287 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:25.282+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:25.413 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:25.407+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:25.445 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:25.440+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:25.519 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:25.513+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:25.714 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:25.708+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:25.922 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:25.916+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:26.295 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:26.291+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:26.457 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:26.452+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:26.460 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:26.453+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:26.500 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:26.494+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:26.740 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:26.735+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:26.946 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:26.940+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:27.319 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:27.313+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:27.462 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:27.457+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:27.466 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:27.460+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:27.473 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:27.468+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:27.731 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:27.727+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:27.761 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:27.756+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:27.995 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:27.985+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:28.299 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:28.294+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:28.451 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:28.446+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:28.453 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:28.448+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:28.462 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:28.457+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:28.687 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:28.683+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:28.758 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:28.753+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:28.954 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:28.949+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:29.273 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:29.268+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:29.410 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:29.405+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:29.433 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:29.426+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:29.507 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:29.503+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:29.723 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:29.717+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:29.737 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:29.731+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:29.985 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:29.979+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:30.309 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:30.305+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:30.444 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:30.439+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:30.451 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:30.445+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:30.508 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:30.502+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:30.700 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:30.694+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:30.740 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:30.735+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:31.001 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:30.995+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:31.293 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:31.287+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:31.426 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:31.420+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:31.428 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:31.422+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:31.553 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:31.548+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:31.700 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:31.695+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:31.790 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:31.786+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:32.052 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:32.045+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:32.298 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:32.294+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:32.425 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:32.419+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:32.473 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:32.468+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:32.524 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:32.518+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:32.707 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:32.701+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:32.762 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:32.756+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:33.087 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:33.081+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:33.315 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:33.311+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:33.398 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:33.394+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:33.523 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:33.518+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:33.527 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:33.523+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:33.717 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:33.713+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:33.744 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:33.739+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:34.051 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:34.045+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:34.326 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:34.321+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:34.430 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:34.423+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:34.495 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:34.489+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:34.577 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:34.572+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:34.738 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:34.733+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:34.742 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:34.738+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:35.090 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:35.084+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:35.314 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:35.309+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:35.431 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:35.425+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:35.494 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:35.488+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:35.547 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:35.541+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:35.712 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:35.708+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:35.743 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:35.739+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:36.148 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:36.139+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:36.321 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:36.317+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:36.403 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:36.397+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:36.488 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:36.482+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:36.518 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:36.513+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:36.715 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:36.710+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:36.724 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:36.719+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:37.167 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:37.162+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:37.295 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:37.290+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:37.440 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:37.434+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:37.503 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:37.495+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:37.528 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:37.523+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:37.717 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:37.712+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:37.721 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:37.715+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:38.169 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:38.163+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:38.252 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:38.249+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:38.396 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:38.391+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:38.519 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:38.514+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:38.528 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:38.523+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:38.688 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:38.684+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:38.690 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:38.686+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:39.168 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:39.161+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:39.267 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:39.261+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:39.431 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:39.425+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:39.503 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:39.498+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:39.511 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:39.506+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:39.694 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:39.690+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:39.700 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:39.696+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:40.149 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:40.142+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:40.307 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:40.302+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:40.434 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:40.427+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:40.500 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:40.484+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:40.533 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:40.528+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:40.654 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:40.650+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:40.715 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:40.711+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:41.165 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:41.160+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:41.359 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:41.354+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:41.410 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:41.404+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:41.506 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:41.502+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:41.573 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:41.568+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:41.617 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:41.613+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:41.703 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:41.699+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:42.169 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:42.163+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:42.369 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:42.364+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:42.393 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:42.387+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:42.530 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:42.525+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:42.577 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:42.572+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:42.611 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:42.600+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:42.732 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:42.727+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:43.200 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:43.193+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:43.383 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:43.378+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:43.394 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:43.390+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:43.492 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:43.487+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:43.610 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:43.606+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:43.670 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:43.662+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:43.760 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:43.756+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:44.191 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:44.185+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:44.394 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:44.389+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:44.412 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:44.407+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:44.534 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:44.530+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:44.619 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:44.614+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:44.637 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:44.631+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:44.818 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:44.811+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:45.193 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:45.182+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:45.429 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:45.423+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:45.447 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:45.443+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:45.505 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:45.499+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:45.624 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:45.616+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:45.653 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:45.647+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:45.842 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:45.836+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:46.199 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:46.193+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:46.424 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:46.418+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:46.492 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:46.486+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:46.508 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:46.500+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:46.636 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:46.629+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:46.657 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:46.651+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:46.865 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:46.860+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:47.175 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:47.170+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:47.454 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:47.448+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:47.532 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:47.525+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:47.547 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:47.541+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:47.613 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:47.607+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:47.642 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:47.636+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:47.823 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:47.816+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:48.160 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:48.153+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:48.421 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:48.415+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:48.552 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:48.546+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:48.585 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:48.579+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:48.588 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:48.582+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:48.634 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:48.628+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:48.862 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:48.857+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:49.188 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:49.182+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:49.458 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:49.453+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:49.540 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:49.535+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:49.541 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:49.537+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:49.590 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:49.586+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:49.617 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:49.611+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:49.832 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:49.828+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:50.192 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:50.186+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:50.443 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:50.437+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:50.572 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:50.566+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:50.581 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:50.577+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:50.590 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:50.585+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:50.630 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:50.625+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:50.851 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:50.847+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:51.221 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:51.214+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:51.467 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:51.461+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:51.618 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:51.614+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:51.619 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:51.613+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:51.622 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:51.618+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:51.651 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:51.645+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:51.894 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:51.889+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:52.220 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:52.214+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:52.497 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:52.491+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:52.600 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:52.595+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:52.607 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:52.602+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:52.608 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:52.603+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:52.663 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:52.657+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:52.903 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:52.898+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:53.236 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:53.230+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:53.386 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:58:53.382+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 41sec e263 v263) 2019-12-09T15:58:53.465 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:53.458+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:53.618 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:53.613+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:53.641 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:53.636+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:53.651 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:53.646+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:53.655 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:53.650+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:53.876 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:53.868+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:54.279 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:54.273+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:54.424 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:54.418+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:54.579 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:54.574+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:54.645 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:54.641+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:54.660 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:54.654+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:54.694 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:54.690+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:54.860 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:54.855+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:55.238 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:55.232+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:55.388 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:55.381+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:55.559 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:55.552+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:55.651 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:55.646+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:55.667 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:55.663+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:55.697 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:55.693+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:55.818 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:55.812+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:56.279 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:56.273+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:56.436 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:56.431+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:56.610 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:56.605+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:56.652 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:56.645+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:56.718 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:56.713+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:56.756 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:56.746+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:56.852 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:56.847+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:57.242 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:57.237+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:57.487 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:57.480+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:57.600 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:57.595+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:57.650 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:57.642+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:57.740 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:57.736+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:57.742 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:57.738+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:57.844 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:57.839+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:58.292 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:58.287+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:58.408 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:58:58.402+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 41sec e263 v263) 2019-12-09T15:58:58.457 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:58.452+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:58.570 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:58.565+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:58.654 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:58.649+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:58.705 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:58.699+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:58.791 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:58.786+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:58:58.809 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:58.803+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:59.291 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:58:59.285+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:58:59.477 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:58:59.471+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:58:59.571 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:58:59.565+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:58:59.662 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:58:59.657+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:58:59.748 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:58:59.744+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:58:59.802 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:58:59.798+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:58:59.816 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:58:59.812+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:59:00.260 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:59:00.255+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:59:00.494 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:59:00.488+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:59:00.572 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:59:00.567+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:59:00.680 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:59:00.674+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:59:00.767 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:59:00.762+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:59:00.793 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:59:00.788+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:59:00.816 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:59:00.810+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:59:01.292 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:59:01.285+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:59:01.483 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:59:01.477+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:59:01.611 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:59:01.606+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:59:01.644 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:59:01.639+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:59:01.795 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:59:01.791+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:59:01.805 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:59:01.800+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:59:01.837 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:59:01.832+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:59:02.255 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:59:02.249+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:59:02.461 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:59:02.455+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:59:02.581 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:59:02.575+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:59:02.632 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T15:59:02.626+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.849157+0000 front 2019-12-09T15:57:38.563374+0000 (oldest deadline 2019-12-09T15:58:21.388962+0000) 2019-12-09T15:59:02.805 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:59:02.800+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:59:02.854 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:59:02.849+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:59:02.856 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:59:02.852+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T15:59:03.258 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T15:59:03.253+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:41.524269+0000 front 2019-12-09T15:57:37.979262+0000 (oldest deadline 2019-12-09T15:58:21.199410+0000) 2019-12-09T15:59:03.442 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T15:59:03.437+0000 1694f700 -1 osd.5 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:42.573326+0000 front 2019-12-09T15:57:41.117960+0000 (oldest deadline 2019-12-09T15:58:22.693023+0000) 2019-12-09T15:59:03.503 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T15:59:03.492+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 41sec e263 v263) 2019-12-09T15:59:03.540 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T15:59:03.536+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.717044+0000 front 2019-12-09T15:57:42.575892+0000 (oldest deadline 2019-12-09T15:58:21.073607+0000) 2019-12-09T15:59:03.782 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T15:59:03.779+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:37.977957+0000 front 2019-12-09T15:57:38.864359+0000 (oldest deadline 2019-12-09T15:58:27.708106+0000) 2019-12-09T15:59:03.824 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T15:59:03.819+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:40.662257+0000 front 2019-12-09T15:57:41.881896+0000 (oldest deadline 2019-12-09T15:58:21.830723+0000) 2019-12-09T15:59:03.871 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T15:59:03.867+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T15:57:38.518466+0000 front 2019-12-09T15:57:38.523169+0000 (oldest deadline 2019-12-09T15:58:19.427059+0000) 2019-12-09T16:00:00.538 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:00.531+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:01.119 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:01.113+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:01.514 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:01.508+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:01.568 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:01.563+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:02.125 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:02.119+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:02.519 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:02.510+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:02.609 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:02.603+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:03.170 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:03.162+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:03.483 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:03.477+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:03.607 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:03.601+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:03.854 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:03.849+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:04.066 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:04.060+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:04.180 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:04.173+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:04.484 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:04.477+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:04.583 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:04.576+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:04.829 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:04.825+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:05.084 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:05.079+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:05.184 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:05.177+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:05.480 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:05.472+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:05.654 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:05.641+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:05.806 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:05.800+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:05.915 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:05.907+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:06.109 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:06.104+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:06.109 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:06.105+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:06.235 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:06.229+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:06.533 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:06.527+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:06.649 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:06.643+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:06.776 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:06.771+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:06.900 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:06.893+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:07.078 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:07.073+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:07.080 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:07.077+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:07.256 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:07.250+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:07.571 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:07.565+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:07.617 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:07.610+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:07.738 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:07.731+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:07.873 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:07.866+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:08.123 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:08.117+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:08.124 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:08.119+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:08.279 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:08.272+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:08.604 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:08.597+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:08.609 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:08.603+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:08.696 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:08.692+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:08.909 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:08.902+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:09.140 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:09.133+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:09.140 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:09.136+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:09.258 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:09.251+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:09.572 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:09.564+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:09.574 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:09.568+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:09.722 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:09.716+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:09.910 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:09.903+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:10.140 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:10.135+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:10.141 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:10.137+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:10.224 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:10.217+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:10.575 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:10.569+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:10.576 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:10.571+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:10.680 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:10.674+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:10.944 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:10.937+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:11.171 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:11.165+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:11.171 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:11.167+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:11.220 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:11.211+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:11.585 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:11.580+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:11.630 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:11.625+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:11.706 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:11.701+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:11.945 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:11.938+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:12.217 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:12.209+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:12.218 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:12.214+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:12.250 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:12.244+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:12.580 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:12.574+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:12.632 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:12.625+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:12.728 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:12.723+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:12.927 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:12.920+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:13.205 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:13.192+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:13.206 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:13.201+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:13.208 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:13.201+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:13.593 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:13.588+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:13.630 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:13.624+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:13.697 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:13.691+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:13.725 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:13.717+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 46 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:13.975 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:13.969+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:14.229 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:14.223+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:14.229 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:14.225+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:14.245 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:14.238+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:14.594 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:14.588+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:14.604 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:14.598+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:14.671 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:14.665+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:14.791 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:14.772+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:14.989 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:14.982+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:15.226 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:15.220+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:15.226 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:15.222+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:15.229 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:15.222+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:15.620 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:15.615+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:15.641 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:15.634+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:15.718 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:15.711+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:15.840 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:15.830+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:15.947 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:15.939+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:16.220 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:16.214+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:16.221 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:16.216+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:16.268 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:16.260+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:16.645 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:16.641+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:16.651 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:16.644+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:16.674 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:16.669+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:16.867 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:16.855+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:17.009 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:17.001+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:17.234 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:17.227+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:17.266 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:17.260+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:17.269 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:17.266+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:17.610 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:17.604+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:17.648 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:17.641+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:17.723 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:17.719+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:17.887 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:17.872+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:17.983 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:17.977+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:18.241 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:18.234+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:18.293 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:18.288+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:18.294 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:18.290+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:18.581 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:18.575+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:18.689 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:18.684+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:18.695 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:18.689+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:18.886 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:18.864+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:19.034 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:19.027+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:19.279 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:19.274+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:19.280 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:19.276+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:19.288 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:19.281+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:19.620 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:19.613+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:19.673 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:19.669+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:19.698 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:19.694+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:19.936 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:19.916+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:20.042 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:20.035+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:20.265 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:20.258+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:20.265 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:20.261+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:20.296 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:20.289+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:20.626 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:20.618+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:20.639 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:20.634+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:20.748 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:20.742+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:20.920 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:20.897+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:20.934 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:20.916+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:21.025 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:21.018+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:21.253 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:21.245+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:21.253 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:21.249+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:21.321 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:21.313+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:21.593 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:21.589+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:21.670 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:21.663+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:21.754 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:21.748+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:21.899 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:21.878+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:21.921 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:21.902+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:22.020 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:22.013+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:22.245 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:22.238+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:22.249 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:22.244+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:22.302 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:22.295+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:22.643 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:22.637+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:22.663 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:22.657+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:22.794 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:22.787+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:22.959 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:22.952+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:22.986 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:22.979+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:22.990 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:22.972+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:23.205 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:23.196+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:23.205 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:23.199+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:23.249 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:47:59.513 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:23.258 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:47:59.522 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:23.265 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:23.259+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:23.285 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:47:59.507 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:23.294 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:47:59.516 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:23.343 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:47:59.565 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:23.346 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:47:59.568 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:23.584 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:47:59.849 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:23.620 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:47:59.843 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:23.673 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:23.668+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:23.694 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:23.689+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:23.710 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:47:59.932 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:23.789 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:23.783+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:23.968 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:23.961+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:24.015 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:23.997+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:24.023 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:24.010+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:24.193 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:24.178+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:24.200 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:24.180+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:24.280 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:24.273+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:24.638 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:24.632+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:24.748 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:24.743+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:24.757 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:24.751+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:24.979 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:24.973+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:25.020 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:24.998+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:25.035 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:25.016+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:25.199 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:25.193+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:25.202 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:25.195+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:25.295 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:25.289+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:25.619 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:25.613+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:25.728 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:25.723+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:25.799 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:25.794+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:25.940 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:25.935+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:26.013 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:26.007+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:26.047 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:26.029+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:26.229 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:26.224+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:26.230 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:26.226+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:26.336 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:26.330+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:26.647 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:26.641+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:26.757 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:26.753+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:26.773 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:26.768+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:26.988 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:26.982+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:27.029 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:27.023+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:27.057 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:27.042+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:27.200 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:27.194+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:27.201 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:27.197+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:27.343 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:27.337+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:27.621 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:27.615+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:27.721 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:27.715+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:27.752 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:27.747+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:28.010 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:28.004+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:28.044 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:28.039+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:28.067 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:28.052+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:28.254 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:28.248+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:28.258 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:28.254+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:28.328 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:28.316+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:28.576 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:28.570+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:28.679 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:28.673+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:28.795 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:28.790+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:29.001 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:28.987+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:29.101 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:29.095+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:29.132 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:29.117+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:29.263 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:29.256+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:29.264 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:29.259+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:29.375 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:29.370+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:29.560 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:29.552+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:29.690 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:29.685+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:29.796 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:29.791+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:30.046 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:30.039+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:30.114 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:30.091+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:30.124 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:30.110+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:30.301 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:30.295+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:30.302 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:30.297+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:30.375 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:30.370+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:30.544 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:30.538+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:30.658 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:30.652+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:30.775 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:30.771+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:31.014 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:31.009+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:31.091 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:31.073+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:31.106 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:31.087+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:31.286 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:31.279+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:17.847605+0000 front 2019-12-09T15:59:17.937165+0000 (oldest deadline 2019-12-09T16:00:03.115971+0000) 2019-12-09T16:00:31.286 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:31.282+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:31.378 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:31.371+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:31.559 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:31.554+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:31.710 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:31.706+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:31.801 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:31.796+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:31.988 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:31.982+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:32.136 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:32.131+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:32.167 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:32.152+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:32.308 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:32.304+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:32.407 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:32.401+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:32.575 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:32.569+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:32.693 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:32.674+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:32.799 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:32.795+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:32.971 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:32.965+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:33.216 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:33.193+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:33.222 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:33.210+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:33.330 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:33.325+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:33.429 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:33.423+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:33.578 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:33.572+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:33.686 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:33.680+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:33.795 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:33.788+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:33.943 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:33.937+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:34.198 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:34.182+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:34.204 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:00:34.198+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e269 v269) 2019-12-09T16:00:34.211 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:34.195+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:34.335 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:34.331+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:34.401 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:34.395+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:34.561 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:34.555+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:34.654 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:34.649+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:34.789 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:34.783+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:34.952 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:34.942+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:35.182 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:35.158+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:35.197 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:35.178+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:35.380 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:35.362+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:35.403 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:35.397+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:35.533 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:35.527+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:35.636 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:35.632+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:35.798 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:35.793+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:35.921 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:35.914+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:36.156 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:36.133+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:36.171 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:36.153+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:36.334 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:36.330+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:36.389 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:36.383+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:36.571 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:36.565+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:36.669 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:36.664+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:36.802 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:36.795+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:36.956 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:36.949+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:37.193 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:37.171+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:37.211 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:37.190+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:37.296 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:37.291+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:37.348 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:37.341+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:37.622 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:37.616+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:37.681 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:37.674+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:37.841 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:37.836+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:37.970 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:37.964+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:38.180 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:38.155+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:38.193 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:38.177+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:38.258 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:38.253+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:38.386 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:38.380+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:38.641 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:38.635+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:38.718 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:38.713+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:38.874 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:38.868+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:38.934 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:38.926+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:39.171 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:39.165+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:39.210 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:39.187+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:39.223 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:00:39.219+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e269 v269) 2019-12-09T16:00:39.285 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:39.280+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:39.400 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:39.394+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:39.662 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:39.656+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:39.750 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:39.745+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:39.925 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:39.919+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:39.960 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:39.954+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:40.185 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:40.159+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:40.194 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:40.178+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:40.243 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:40.238+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:40.453 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:40.448+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:40.679 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:40.674+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:40.776 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:40.770+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:40.919 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:40.914+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:40.952 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:40.946+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:41.168 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:41.145+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:41.184 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:41.165+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:41.203 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:41.198+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:41.497 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:41.492+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:41.707 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:41.701+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:41.789 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:41.784+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:41.970 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:41.964+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:41.996 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:41.989+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:42.170 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:42.145+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:42.171 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:42.166+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:42.189 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:42.165+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:42.546 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:42.539+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:42.694 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:42.687+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:42.789 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:42.783+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:42.961 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:42.953+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:43.000 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:42.994+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:43.125 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:43.118+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:43.160 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:43.139+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:43.205 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:43.201+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:43.564 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:43.557+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:43.705 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:43.698+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:43.821 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:43.816+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:43.943 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:43.936+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:44.042 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:44.035+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:44.152 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:44.129+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:44.166 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:44.161+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:44.167 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:44.148+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:44.243 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:00:44.239+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e269 v269) 2019-12-09T16:00:44.526 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:44.519+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:44.717 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:44.710+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:44.871 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:44.866+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:44.908 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:44.900+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:45.006 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:45.001+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:45.161 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:45.138+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:45.177 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:45.156+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:45.194 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:45.189+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:45.547 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:45.540+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:45.689 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:45.682+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:45.826 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:45.822+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:45.933 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:45.927+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:45.965 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:45.960+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:46.174 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:46.150+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:46.182 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:46.177+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:46.189 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:46.170+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:46.546 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:46.539+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:46.729 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:46.721+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:46.859 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:46.854+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:46.928 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:46.921+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:46.997 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:46.991+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:47.169 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:47.146+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:47.179 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:47.175+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:47.183 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:47.165+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:47.533 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:47.525+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:47.709 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:47.702+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:47.866 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:47.861+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:47.890 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:47.883+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:47.988 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:47.984+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:48.159 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:48.153+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:48.207 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:48.201+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:48.239 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:48.224+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:48.501 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:48.493+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:48.703 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:48.696+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:48.839 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:48.835+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:48.915 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:48.909+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:48.961 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:48.956+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:49.206 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:49.201+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:49.276 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:00:49.270+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e269 v269) 2019-12-09T16:00:49.290 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:49.265+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:49.312 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:49.281+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:49.517 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:49.511+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:49.752 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:49.746+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:49.851 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:49.844+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:49.934 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:49.928+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:49.936 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:49.929+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:50.249 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:50.245+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:50.284 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:50.270+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:50.294 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:50.280+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:50.559 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:50.553+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:50.778 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:50.770+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:50.840 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:50.834+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:50.963 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:50.956+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:50.970 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:50.960+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:51.253 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:51.248+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:51.327 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:51.303+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:51.340 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:51.322+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:51.565 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:51.558+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:51.801 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:51.795+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:51.804 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:51.799+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:51.941 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:51.936+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:51.948 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:51.941+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:52.214 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:52.209+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:52.344 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:52.319+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:52.369 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:52.340+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:52.546 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:52.535+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:52.757 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:52.750+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:52.829 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:52.823+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:52.981 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:52.976+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:52.983 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:52.977+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:53.251 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:53.246+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:53.354 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:53.330+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:53.370 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:53.349+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:53.584 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:53.577+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:53.799 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:53.792+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:53.871 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:53.864+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:54.008 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:54.004+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:54.010 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:54.003+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:54.284 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:54.280+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:54.298 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:00:54.292+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e269 v269) 2019-12-09T16:00:54.392 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:54.369+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:54.401 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:54.387+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:54.628 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:54.621+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:54.767 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:54.760+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:54.924 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:54.918+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:54.989 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:54.982+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:55.017 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:55.011+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:55.252 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:55.247+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:55.450 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:55.425+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:55.466 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:55.442+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:55.636 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:55.629+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:55.773 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:55.767+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:55.900 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:55.894+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:55.948 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:55.941+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:56.039 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:56.035+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:56.271 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:56.266+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:56.485 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:56.461+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:56.500 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:56.480+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:56.619 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:56.611+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:56.824 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:56.817+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:56.858 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:56.852+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:56.922 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:56.915+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:57.048 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:57.042+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:57.288 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:57.283+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:57.509 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:57.503+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:57.541 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:57.525+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:57.666 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:57.653+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:57.788 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:57.782+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:57.881 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:57.877+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:57.912 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:57.906+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:58.093 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:58.087+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:58.293 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:58.289+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:58.530 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:58.524+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T15:59:40.317866+0000 (oldest deadline 2019-12-09T16:00:20.317866+0000) 2019-12-09T16:00:58.564 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:58.545+0000 1694f700 -1 osd.6 269 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:00:58.640 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:58.632+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:00:58.748 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:00:58.741+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.723350+0000 front 2019-12-09T15:59:18.256397+0000 (oldest deadline 2019-12-09T16:00:00.504617+0000) 2019-12-09T16:00:58.897 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:00:58.884+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:20.373558+0000 front 2019-12-09T15:59:20.063830+0000 (oldest deadline 2019-12-09T16:00:05.200573+0000) 2019-12-09T16:00:58.902 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:00:58.899+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:16.105177+0000 front 2019-12-09T15:59:16.137509+0000 (oldest deadline 2019-12-09T16:00:01.366435+0000) 2019-12-09T16:00:59.081 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:00:59.075+0000 1694f700 -1 osd.2 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:18.758978+0000 front 2019-12-09T15:59:19.135597+0000 (oldest deadline 2019-12-09T16:00:03.178351+0000) 2019-12-09T16:00:59.303 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:00:59.298+0000 1694f700 -1 osd.0 269 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:00:59.374 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:00:59.365+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e269 v269) 2019-12-09T16:00:59.642 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:00:59.636+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T15:59:15.924878+0000 front 2019-12-09T15:59:15.695321+0000 (oldest deadline 2019-12-09T16:00:00.915331+0000) 2019-12-09T16:01:00.153 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:00:59.990+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 166 slow ops, oldest is osd_op(client.4355.0:111687 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e267) 2019-12-09T16:01:00.396 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:01:00.327+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:01:01.464 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:01:01.455+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116066 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418862224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-09T16:01:02.315 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:01:02.299+0000 20e25700 -1 RGWWatcher::handle_error cookie 5418862224 err (107) Transport endpoint is not connected 2019-12-09T16:02:26.553 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:26.548+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:26.633 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:26.622+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:26.705 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:26.701+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:27.600 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:27.594+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:27.613 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:27.607+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:27.715 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:27.710+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:28.286 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:28.281+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:28.407 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:28.402+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:28.611 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:28.606+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:28.644 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:28.637+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:28.699 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:28.692+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:28.709 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:28.705+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:29.242 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:29.237+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:29.244 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:29.239+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:29.386 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:29.381+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:29.618 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:29.612+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:29.639 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:29.634+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:29.694 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:29.690+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:29.763 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:29.757+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:30.211 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:30.203+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:30.243 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:30.239+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:30.370 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:30.364+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:30.647 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:30.641+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:30.681 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:30.676+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:30.695 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:30.690+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:30.813 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:30.809+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:31.208 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:31.203+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:31.287 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:31.282+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:31.382 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:31.376+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:31.641 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:31.634+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:31.664 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:31.658+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:31.685 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:31.681+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:31.775 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:31.770+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:32.253 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:32.247+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:32.264 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:32.251+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:32.357 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:32.352+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:32.628 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:32.624+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:32.652 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:32.647+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:32.652 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:32.649+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:32.751 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:32.746+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:33.277 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:33.272+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:33.307 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:33.301+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:33.360 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:33.355+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:33.629 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:33.620+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:33.679 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:33.673+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:33.700 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:33.694+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:33.743 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:33.739+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:34.307 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:34.303+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:34.345 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:34.340+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:34.378 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:34.370+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:34.657 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:34.653+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:34.658 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:34.654+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:34.714 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:34.708+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:34.741 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:34.736+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:35.292 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:35.287+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:35.334 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:35.329+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:35.377 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:35.372+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:35.649 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:35.645+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:35.668 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:35.663+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:35.724 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:35.719+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:35.754 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:35.749+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:36.273 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:36.267+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:36.338 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:36.330+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:36.387 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:36.381+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:36.623 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:36.618+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:36.628 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:36.619+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:36.690 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:36.685+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:36.740 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:36.734+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:37.264 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:37.259+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:37.386 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:37.378+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:37.389 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:37.384+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:37.663 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:37.659+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:37.676 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:37.672+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:37.696 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:37.690+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:37.722 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:37.717+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:38.260 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:38.255+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:38.403 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:38.397+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:38.439 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:38.434+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:38.663 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:38.659+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:38.718 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:38.713+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:38.719 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:38.714+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:38.741 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:38.735+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:39.282 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:39.278+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:39.415 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:39.409+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:39.445 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:39.439+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:39.680 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:39.675+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:39.695 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:39.690+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:39.724 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:39.719+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:39.732 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:39.727+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:40.307 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:40.303+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:40.411 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:40.406+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:40.468 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:40.463+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:40.670 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:40.664+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:40.707 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:40.701+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:40.710 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:40.703+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:40.715 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:40.712+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:41.278 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:41.273+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:41.465 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:41.459+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:41.492 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:41.485+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:41.697 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:41.692+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:41.708 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:41.703+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:41.729 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:41.723+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:41.761 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:41.756+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:42.261 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:42.253+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:42.421 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:42.415+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:42.496 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:42.488+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:42.668 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:42.662+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:42.713 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:42.708+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:42.755 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:42.749+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:42.809 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:42.804+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:43.229 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:43.224+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:43.472 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:43.467+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:43.521 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:43.515+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:43.677 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:43.671+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:43.690 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:43.684+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:43.776 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:43.771+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:43.777 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:43.769+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:44.253 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:44.248+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:44.457 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:44.451+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:44.550 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:44.543+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:44.670 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:44.665+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:44.684 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:44.679+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:44.731 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:44.726+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:44.744 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:44.738+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:45.246 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:45.242+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:45.425 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:45.418+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:45.525 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:45.519+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:45.664 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:45.659+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:45.667 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:45.662+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:45.713 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:45.708+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:45.768 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:45.763+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:46.240 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:46.235+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:46.405 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:46.399+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:46.513 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:46.502+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:46.625 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:46.620+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:46.688 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:46.683+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:46.749 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:46.744+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:46.785 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:46.775+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:47.200 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:47.194+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:47.447 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:47.441+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:47.491 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:47.486+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:47.657 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:47.651+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:47.663 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:47.657+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:47.766 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:47.760+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:47.769 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:47.763+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:48.245 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:48.239+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:48.437 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:48.431+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:48.505 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:48.500+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:48.632 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:48.626+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:48.638 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:48.633+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:48.755 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:48.749+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:48.822 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:48.812+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:49.294 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:49.288+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:49.430 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:49.424+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:49.543 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:49.536+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:49.616 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:49.609+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:49.629 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:49.622+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:49.776 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:49.770+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:49.792 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:49.786+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:50.256 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:50.250+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:50.451 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:50.445+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:50.580 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:50.574+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:50.630 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:50.625+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:50.650 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:50.646+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:50.767 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:50.762+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:50.817 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:50.812+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:51.238 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:51.233+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:51.437 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:51.430+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:51.612 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:51.607+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:51.620 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:51.612+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:51.634 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:51.628+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:51.786 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:51.780+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:51.871 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:51.866+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:52.250 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:52.245+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:52.461 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:52.454+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:52.607 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:52.600+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:52.608 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:52.602+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:52.648 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:52.643+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:52.803 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:52.797+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:52.859 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:52.856+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:53.270 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:53.264+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:53.478 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:53.472+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:53.587 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:53.580+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:53.633 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:53.627+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:53.636 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:53.631+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:53.765 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:53.760+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:53.874 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:53.868+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:54.237 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:54.232+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:54.435 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:54.429+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:54.596 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:54.591+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:54.619 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:54.614+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:54.635 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:54.630+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:54.747 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:54.743+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:54.846 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:54.842+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:55.267 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:55.262+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:55.457 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:55.444+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:55.607 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:55.601+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:55.625 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:55.620+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:55.628 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:55.623+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:55.757 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:55.753+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:55.896 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:55.888+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:56.240 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:56.236+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:56.420 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:56.415+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:56.564 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:56.558+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:56.599 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:56.594+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:56.620 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:56.614+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:56.727 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:56.723+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:56.890 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:56.886+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:57.246 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:57.241+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:57.457 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:57.453+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:57.553 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:57.547+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:57.578 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:57.572+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:57.656 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:57.651+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:57.789 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:57.776+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:57.865 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:57.861+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:58.241 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:58.237+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:58.464 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:58.458+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:58.549 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:58.545+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:58.613 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:58.609+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:58.618 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:58.613+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:58.759 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:58.754+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:58.835 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:58.830+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:59.245 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:02:59.240+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:02:59.516 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:02:59.511+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:02:59.578 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:02:59.572+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:02:59.587 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:02:59.582+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:02:59.675 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:02:59.668+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:02:59.769 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:02:59.764+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:02:59.834 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:02:59.829+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:02:59.848 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:02:59.843+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 45sec e274 v274) 2019-12-09T16:03:00.253 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:03:00.249+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:03:00.533 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:03:00.528+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:03:00.609 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:03:00.604+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:03:00.626 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:03:00.620+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:03:00.654 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:03:00.649+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:03:00.732 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:03:00.726+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:03:00.858 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:03:00.852+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:03:01.261 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:03:01.257+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:03:01.579 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:03:01.575+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:03:01.613 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:03:01.608+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:03:01.660 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:03:01.655+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:03:01.693 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:03:01.688+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:03:01.774 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:03:01.769+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:03:01.892 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:03:01.887+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:03:02.221 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:03:02.216+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:03:02.586 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:03:02.581+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:03:02.628 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:03:02.618+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:03:02.660 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:03:02.656+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:03:02.668 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:03:02.664+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:03:02.781 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:03:02.776+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:03:02.930 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:03:02.925+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:03:03.230 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:03:03.222+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:03:03.557 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:03:03.552+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:03:03.607 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:03:03.602+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:03:03.617 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:03:03.613+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:03:03.665 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:03:03.660+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:03:03.763 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:03:03.758+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:03:03.963 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:03:03.956+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:03:04.235 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:03:04.230+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:03:04.607 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:03:04.604+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:03:04.656 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:03:04.652+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:03:04.657 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:03:04.652+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:03:04.706 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:03:04.699+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:03:04.737 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:03:04.732+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:03:04.861 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:03:04.857+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 45sec e274 v274) 2019-12-09T16:03:04.943 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:03:04.939+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:03:05.268 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:03:05.262+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:03:05.622 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:03:05.617+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:03:05.623 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:03:05.618+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:03:05.641 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:03:05.636+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:03:05.686 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:03:05.681+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:03:05.716 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:03:05.711+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:03:05.986 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:03:05.981+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:03:06.277 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:03:06.273+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:03:06.609 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:03:06.604+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:03:06.615 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:03:06.611+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:03:06.661 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:03:06.656+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:03:06.675 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:03:06.670+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:03:06.732 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:03:06.727+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:03:06.977 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:03:06.971+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:03:07.300 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:03:07.292+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:03:07.595 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:03:07.591+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:03:07.621 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:03:07.615+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:03:07.633 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:03:07.623+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:03:07.722 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:03:07.717+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:03:07.735 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:03:07.730+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:03:08.009 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:03:08.004+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:03:08.271 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:03:08.265+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:03:08.563 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:03:08.558+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:03:08.594 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:03:08.589+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:03:08.638 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:03:08.633+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:03:08.744 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:03:08.735+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:03:08.757 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:03:08.752+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:03:08.968 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:03:08.963+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:03:09.282 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:03:09.276+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:03:09.567 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:03:09.563+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:47.125834+0000 front 2019-12-09T16:01:44.878419+0000 (oldest deadline 2019-12-09T16:02:25.865785+0000) 2019-12-09T16:03:09.570 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:03:09.566+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:41.207798+0000 front 2019-12-09T16:01:46.925563+0000 (oldest deadline 2019-12-09T16:02:26.150704+0000) 2019-12-09T16:03:09.610 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:03:09.605+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:45.053824+0000 front 2019-12-09T16:01:46.184835+0000 (oldest deadline 2019-12-09T16:02:29.207674+0000) 2019-12-09T16:03:09.769 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:03:09.764+0000 1694f700 -1 osd.5 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.220804+0000 front 2019-12-09T16:01:45.057408+0000 (oldest deadline 2019-12-09T16:02:27.623496+0000) 2019-12-09T16:03:09.798 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:03:09.794+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.168587+0000 front 2019-12-09T16:01:45.894258+0000 (oldest deadline 2019-12-09T16:02:28.062244+0000) 2019-12-09T16:03:09.952 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:03:09.946+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 45sec e274 v274) 2019-12-09T16:03:10.001 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:03:09.997+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:46.987430+0000 front 2019-12-09T16:01:46.181999+0000 (oldest deadline 2019-12-09T16:02:26.696103+0000) 2019-12-09T16:03:10.234 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:03:10.230+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:01:44.878423+0000 front 2019-12-09T16:01:45.883076+0000 (oldest deadline 2019-12-09T16:02:27.722517+0000) 2019-12-09T16:03:35.243 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:51:11.507 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:03:35.269 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:51:11.533 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:03:35.312 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:51:11.534 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:03:35.315 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:51:11.537 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:03:35.330 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:51:11.552 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:03:35.335 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:51:11.557 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:03:35.487 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:51:11.709 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:03:35.509 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:51:11.773 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:03:35.575 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:51:11.797 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:04:07.221 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:07.215+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:08.197 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:08.193+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:08.382 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:08.374+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:08.520 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:08.516+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:08.866 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:08.856+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:09.219 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:09.214+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:09.355 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:09.348+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:09.522 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:09.515+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:09.850 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:09.843+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:10.003 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:09.996+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:10.183 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:10.177+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:10.187 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:10.183+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:10.400 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:10.393+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:10.517 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:10.513+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:10.623 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:10.616+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:10.826 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:10.819+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:11.029 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:11.024+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:11.236 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:11.230+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:11.237 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:11.233+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:11.385 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:11.380+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:11.561 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:11.556+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:11.596 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:11.588+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:11.836 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:11.829+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:12.037 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:12.032+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:12.266 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:12.259+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:12.272 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:12.265+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:12.368 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:12.362+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:12.571 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:12.566+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:12.604 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:12.598+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:12.801 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:12.794+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:13.029 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:13.024+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:13.303 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:13.297+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:13.306 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:13.302+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:13.413 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:13.406+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:13.619 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:13.611+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:13.657 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:13.649+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:13.813 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:13.798+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:14.071 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:14.065+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:14.321 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:14.315+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:14.325 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:14.321+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:14.451 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:14.444+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:14.665 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:14.659+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:14.674 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:14.667+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:14.785 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:14.780+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:15.079 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:15.074+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:15.292 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:15.282+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:15.292 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:15.286+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:15.433 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:15.427+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:15.636 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:15.630+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:15.720 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:15.714+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:15.800 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:15.794+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:16.100 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:16.094+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:16.249 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:16.242+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:16.249 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:16.245+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:16.484 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:16.477+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:16.663 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:16.657+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:16.728 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:16.723+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:16.760 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:16.754+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:17.119 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:17.114+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:17.207 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:17.200+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:17.208 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:17.203+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:17.486 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:17.480+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:17.703 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:17.697+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:17.704 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:17.697+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:17.736 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:17.730+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:18.099 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:18.095+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:18.258 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:18.250+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:18.259 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:18.254+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:18.449 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:18.441+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:18.681 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:18.675+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:18.702 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:18.697+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:18.754 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:18.747+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:19.053 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:19.048+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:19.221 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:19.216+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:19.224 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:19.220+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:19.438 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:19.432+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:19.723 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:19.717+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:19.738 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:19.728+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:19.758 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:19.753+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:20.105 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:20.101+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:20.246 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:20.240+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:20.256 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:20.243+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:20.464 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:20.458+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:20.688 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:20.681+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:20.730 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:20.724+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:20.772 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:20.765+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:21.116 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:21.111+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:21.157 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:21.151+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:21.287 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:21.281+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:21.291 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:21.287+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:21.436 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:21.431+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:21.719 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:21.714+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:21.757 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:21.744+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:21.770 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:21.765+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:22.125 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:22.119+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:22.153 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:22.148+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 24 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:22.297 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:22.290+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:22.298 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:22.294+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:22.442 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:22.436+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:22.606 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:51:58.871 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:04:22.622 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:51:58.886 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:04:22.627 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:51:58.849 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:04:22.633 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:51:58.855 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:04:22.634 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:51:58.856 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:04:22.647 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:51:58.864 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:04:22.712 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:51:58.976 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:04:22.726 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:22.722+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:22.728 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:51:58.950 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:04:22.741 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:51:58.960 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:04:22.766 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:22.759+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:22.768 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:22.764+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:23.141 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:23.136+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:23.166 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:23.154+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 104 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:23.306 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:23.299+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:23.307 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:23.302+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:23.403 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:23.398+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:23.682 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:23.675+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:23.722 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:23.716+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:23.735 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:23.729+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:24.153 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:24.143+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:24.175 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:24.169+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:24.294 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:24.288+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:24.294 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:24.290+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:24.459 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:24.450+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:24.648 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:24.642+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:24.677 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:24.673+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:24.706 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:24.700+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:25.140 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:25.123+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:25.216 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:25.210+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:25.276 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:25.268+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:25.277 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:25.272+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:25.425 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:25.419+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:25.655 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:25.649+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:25.660 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:25.654+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:25.696 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:25.691+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:26.136 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:26.124+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:26.209 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:26.203+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:26.304 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:26.299+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:26.305 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:26.301+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:26.418 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:26.412+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:26.671 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:26.666+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:26.671 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:26.664+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:26.701 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:26.695+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:27.133 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:27.114+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:27.242 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:27.236+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:27.306 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:27.301+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:27.307 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:27.303+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:27.418 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:27.412+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:27.662 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:27.656+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:27.712 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:27.706+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:27.722 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:27.716+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:28.151 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:28.145+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:28.184 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:28.166+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:28.290 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:28.285+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:28.322 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:28.316+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:28.325 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:28.320+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:28.458 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:28.453+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:28.623 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:28.617+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:28.666 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:28.662+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:28.747 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:28.741+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:29.197 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:29.175+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:29.212 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:29.194+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:29.343 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:29.338+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:29.362 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:29.356+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:29.366 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:29.361+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:29.441 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:29.436+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:29.613 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:29.607+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:29.657 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:29.650+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:29.720 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:29.711+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:30.221 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:30.198+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:30.234 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:30.217+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:30.299 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:30.294+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:30.398 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:30.393+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:30.400 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:30.396+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:30.445 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:30.439+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:30.572 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:30.565+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:30.682 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:30.677+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:30.747 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:30.741+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:31.220 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:31.196+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:31.234 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:31.216+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:31.353 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:31.348+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:31.433 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:31.425+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:31.434 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:31.429+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:31.472 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:31.465+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:31.562 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:31.555+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:31.682 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:31.676+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:31.752 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:31.747+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:32.259 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:32.236+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:32.272 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:32.255+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:32.351 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:32.345+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:32.448 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:32.441+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:32.449 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:32.445+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:32.520 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:32.513+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:32.532 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:32.525+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:32.657 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:32.652+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:32.761 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:32.755+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:33.243 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:33.237+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:33.276 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:33.258+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:33.324 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:33.317+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:33.477 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:33.471+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:33.481 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:33.477+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:33.514 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:33.501+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:33.535 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:33.528+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:33.663 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:33.657+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:33.744 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:33.737+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:34.312 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:34.287+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:34.324 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:34.307+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:34.332 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:34.326+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:34.499 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:34.492+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:34.502 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:34.498+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:34.532 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:34.525+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:34.556 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:34.549+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:34.675 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:34.670+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:34.740 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:34.733+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:35.317 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:35.312+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:35.354 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:35.330+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:35.369 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:35.350+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:35.461 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:35.455+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:35.464 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:35.460+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:35.492 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:35.484+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:35.536 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:35.529+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:35.682 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:35.676+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:35.699 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:35.692+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:36.333 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:36.329+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:36.376 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:36.352+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:36.388 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:36.372+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:36.439 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:36.433+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:36.440 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:36.436+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:36.530 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:36.523+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:36.578 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:36.571+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:36.689 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:36.683+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:36.746 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:36.739+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:37.362 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:37.357+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:37.409 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:37.388+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:37.415 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:37.407+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:37.416 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:37.409+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:37.427 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:37.415+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:37.519 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:37.512+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:37.590 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:37.583+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:37.670 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:37.664+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:37.726 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:37.718+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:38.412 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:38.406+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:38.421 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:38.414+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:38.460 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:38.451+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:38.460 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:38.454+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:38.468 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:38.448+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:38.540 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:38.534+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:38.560 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:38.554+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:38.681 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:38.675+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:38.711 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:38.706+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:39.417 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:39.412+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:39.422 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:39.418+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:39.464 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:39.459+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:39.474 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:39.450+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:39.492 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:39.470+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:39.523 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:39.512+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:39.531 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:39.525+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:39.646 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:39.641+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:39.697 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:39.691+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:40.387 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:40.382+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:40.389 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:40.384+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:40.426 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:04:40.419+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e280 v280) 2019-12-09T16:04:40.495 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:40.477+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:40.505 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:40.499+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:40.509 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:40.490+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:40.539 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:40.532+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:40.547 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:40.542+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:40.608 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:40.603+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:40.715 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:40.710+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:41.385 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:41.378+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:41.386 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:41.382+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:41.514 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:41.507+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:41.522 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:41.516+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:41.526 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:41.520+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:41.545 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:41.529+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:41.553 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:41.547+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:41.628 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:41.621+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:41.678 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:41.672+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:42.394 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:42.389+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:42.394 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:42.391+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:42.502 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:42.496+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:42.518 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:42.513+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:42.528 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:42.521+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:42.563 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:42.541+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:42.578 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:42.559+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:42.652 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:42.646+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:42.710 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:42.704+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:43.385 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:43.378+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:43.386 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:43.382+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:43.474 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:43.467+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:43.517 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:43.511+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:43.551 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:43.543+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:43.596 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:43.573+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:43.610 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:43.591+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:43.612 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:43.606+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:43.699 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:43.692+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:44.411 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:44.406+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:44.412 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:44.407+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:44.521 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:44.514+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:44.537 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:44.531+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:44.557 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:44.550+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:44.642 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:44.636+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:44.647 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:44.634+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:44.678 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:44.660+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:44.684 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:44.678+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:45.375 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:45.369+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:45.376 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:45.372+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:45.451 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:04:45.445+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e280 v280) 2019-12-09T16:04:45.496 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:45.490+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:45.581 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:45.573+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:45.595 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:45.585+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:45.605 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:45.600+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:45.685 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:45.679+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:45.719 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:45.704+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:45.728 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:45.714+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:46.384 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:46.375+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:46.385 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:46.380+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:46.493 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:46.486+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:46.565 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:46.553+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:46.632 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:46.619+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:46.640 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:46.634+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:46.645 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:46.638+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:46.726 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:46.707+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:46.734 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:46.721+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:47.377 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:47.367+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:47.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:47.373+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:47.499 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:47.492+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:47.554 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:47.548+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:47.627 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:47.620+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:47.639 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:47.633+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:47.662 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:47.651+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:47.713 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:47.707+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:47.738 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:47.722+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:48.351 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:48.345+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:48.352 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:48.348+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:48.478 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:48.471+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:48.529 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:48.523+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:48.643 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:48.634+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:48.670 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:48.654+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:48.684 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:48.677+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:48.720 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:48.696+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:48.738 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:48.715+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:49.310 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:49.304+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:49.312 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:49.307+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:49.526 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:49.519+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:49.547 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:49.538+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:49.678 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:49.671+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:49.681 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:49.675+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:49.694 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:49.687+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:49.729 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:49.705+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:49.743 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:49.725+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:50.366 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:50.358+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:50.368 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:50.363+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:50.479 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:04:50.469+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e280 v280) 2019-12-09T16:04:50.559 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:50.551+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:50.598 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:50.592+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:50.646 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:50.642+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:50.686 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:50.680+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:50.703 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:50.695+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:50.719 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:50.694+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:50.732 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:50.715+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:51.327 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:51.322+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:51.330 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:51.326+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:51.586 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:51.579+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:51.620 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:51.614+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:51.633 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:51.628+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:51.685 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:51.679+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:51.730 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:51.723+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:51.741 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:51.717+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:51.755 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:51.737+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:52.373 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:52.367+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:52.374 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:52.369+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:52.595 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:52.588+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:52.617 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:52.613+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:52.650 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:52.643+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:52.688 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:52.681+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:52.773 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:52.762+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:52.790 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:52.784+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:52.824 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:52.806+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:53.416 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:53.411+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:53.421 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:53.416+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:53.596 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:53.589+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:53.642 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:53.637+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:53.691 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:53.683+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:53.704 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:53.698+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:53.797 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:53.790+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:53.809 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:53.802+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:53.840 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:53.823+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:54.379 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:54.371+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:54.381 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:54.375+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:54.593 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:54.588+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:54.620 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:54.613+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:54.699 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:54.693+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:54.710 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:54.704+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:54.768 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:54.761+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:54.876 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:54.852+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:54.893 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:54.871+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:55.421 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:55.413+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:55.423 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:55.418+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:55.504 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:04:55.493+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e280 v280) 2019-12-09T16:04:55.557 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:55.550+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:55.580 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:55.571+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:55.673 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:55.667+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:55.681 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:55.675+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:55.751 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:55.746+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:55.855 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:55.832+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:55.860 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:55.850+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:56.434 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:56.428+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:56.439 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:56.430+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:56.541 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:56.535+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:56.588 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:56.581+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:56.629 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:56.622+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:56.669 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:56.664+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:56.760 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:56.753+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:56.875 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:56.852+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:56.887 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:56.871+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:57.477 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:57.469+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:57.480 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:57.474+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:57.567 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:57.562+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:57.631 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:57.624+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:57.655 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:57.649+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:57.659 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:57.652+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:57.754 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:57.747+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:57.927 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:57.904+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:57.942 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:57.923+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:58.505 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:58.500+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:58.513 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:58.504+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:58.620 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:58.614+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:58.652 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:58.645+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:58.665 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:58.660+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:58.679 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:58.672+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:58.721 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:58.713+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:58.974 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:58.949+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:58.988 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:58.969+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:04:59.532 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:59.523+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:04:59.532 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:04:59.527+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:04:59.667 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:04:59.660+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:04:59.677 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:04:59.670+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:04:59.691 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:04:59.683+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:04:59.702 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:04:59.695+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:04:59.716 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:04:59.710+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:04:59.975 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:59.951+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:04:59.990 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:04:59.971+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:05:00.522 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:05:00.514+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e280 v280) 2019-12-09T16:05:00.577 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:00.572+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:05:00.580 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:00.576+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:05:00.651 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:05:00.644+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:05:00.657 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:05:00.649+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:05:00.714 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:05:00.709+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:05:00.719 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:05:00.713+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:05:00.767 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:05:00.761+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:05:00.937 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:05:00.930+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:05:00.969 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:05:00.951+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:05:01.541 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:01.536+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:05:01.544 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:01.539+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:05:01.609 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:05:01.602+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:05:01.673 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:05:01.666+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:05:01.755 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:05:01.750+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:05:01.765 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:05:01.756+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:05:01.791 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:05:01.784+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:05:01.989 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:05:01.982+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:05:02.016 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:05:01.999+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:05:02.542 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:02.537+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:05:02.543 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:02.539+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:05:02.647 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:05:02.640+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:05:02.648 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:05:02.640+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:05:02.761 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:05:02.756+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:05:02.784 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:05:02.778+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:05:02.788 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:05:02.781+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:05:03.056 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:05:03.049+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:05:03.090 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:05:03.070+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:05:03.599 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:03.591+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:05:03.601 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:03.596+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:05:03.603 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:05:03.596+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:05:03.625 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:05:03.614+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:05:03.764 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:05:03.759+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:05:03.777 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:05:03.772+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:05:03.788 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:05:03.779+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:05:04.060 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:05:04.035+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:05:04.075 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:05:04.056+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:05:04.617 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:05:04.610+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:05:04.625 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:04.620+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:05:04.629 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:04.624+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:05:04.671 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:05:04.664+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:05:04.734 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:05:04.729+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.334867+0000 front 2019-12-09T16:03:26.627800+0000 (oldest deadline 2019-12-09T16:04:09.792851+0000) 2019-12-09T16:05:04.823 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:05:04.818+0000 1694f700 -1 osd.2 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:24.314553+0000 front 2019-12-09T16:03:24.313499+0000 (oldest deadline 2019-12-09T16:04:08.497378+0000) 2019-12-09T16:05:04.829 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:05:04.822+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.758964+0000 front 2019-12-09T16:03:26.746390+0000 (oldest deadline 2019-12-09T16:04:08.245599+0000) 2019-12-09T16:05:05.061 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:05:05.054+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:03:47.198253+0000 (oldest deadline 2019-12-09T16:04:27.198253+0000) 2019-12-09T16:05:05.093 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:05:05.075+0000 1694f700 -1 osd.6 280 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:05:05.602 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:05:05.583+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 43sec e280 v280) 2019-12-09T16:05:05.609 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:05.604+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.560870+0000 front 2019-12-09T16:03:27.197601+0000 (oldest deadline 2019-12-09T16:04:07.175047+0000) 2019-12-09T16:05:05.612 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:05.608+0000 1694f700 -1 osd.0 280 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:05:05.661 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:05:05.653+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:25.375128+0000 front 2019-12-09T16:03:25.087931+0000 (oldest deadline 2019-12-09T16:04:08.135304+0000) 2019-12-09T16:05:05.673 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:05:05.667+0000 1694f700 -1 osd.5 280 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:03:26.307727+0000 front 2019-12-09T16:03:26.309598+0000 (oldest deadline 2019-12-09T16:04:09.968230+0000) 2019-12-09T16:05:06.253 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:05:06.120+0000 1694f700 -1 osd.6 281 get_health_metrics reporting 164 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T16:05:06.712 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:06.657+0000 1694f700 -1 osd.0 281 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:05:07.665 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:05:07.653+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:116455 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437833472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-09T16:05:08.195 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:05:08.183+0000 20e25700 -1 RGWWatcher::handle_error cookie 5437833472 err (107) Transport endpoint is not connected 2019-12-09T16:05:10.969 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:52:47.233 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:05:11.023 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:52:47.288 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:05:11.044 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:52:47.266 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:05:11.055 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:52:47.277 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:05:11.068 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:52:47.290 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:05:11.071 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:52:47.293 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:05:11.135 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:52:47.358 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:05:11.150 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:52:47.415 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:05:11.154 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:52:47.376 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:05:34.758 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:05:34.501+0000 1694f700 -1 osd.1 282 get_health_metrics reporting 1 slow ops, oldest is osd_repop(osd.0.0:29278 5.5 e280/273 5:a93a5511:::notify.2:head v 280'220) 2019-12-09T16:05:39.676 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:05:39.669+0000 3a280700 -1 osd.1 285 build_incremental_map_msg missing incremental map 285 2019-12-09T16:05:39.953 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:05:39.787+0000 39a7f700 -1 osd.1 285 build_incremental_map_msg missing incremental map 285 2019-12-09T16:05:40.286 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:05:39.995+0000 3aa81700 -1 osd.1 285 build_incremental_map_msg missing incremental map 285 2019-12-09T16:06:36.632 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:36.626+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:36.661 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:36.655+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:36.957 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:36.952+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:37.637 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:37.631+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:37.645 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:37.639+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:37.931 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:37.926+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:38.265 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:38.259+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:38.613 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:38.609+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:38.651 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:38.645+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:38.685 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:38.678+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:38.945 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:38.939+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:39.191 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:39.185+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:39.306 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:39.300+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:39.632 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:39.627+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:39.682 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:39.676+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:39.699 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:39.693+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:39.935 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:39.930+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:40.187 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:40.183+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:40.309 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:40.302+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:40.648 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:40.643+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:40.665 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:40.661+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:40.739 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:40.733+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:40.758 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:40.753+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:40.987 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:40.980+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:41.218 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:41.213+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:41.343 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:41.339+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:41.666 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:41.660+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:41.685 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:41.681+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:41.754 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:41.747+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:41.790 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:41.784+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:41.964 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:41.959+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:42.252 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:42.246+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:42.372 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:42.365+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:42.625 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:42.620+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:42.670 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:42.665+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:42.715 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:42.709+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:42.793 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:42.788+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:42.987 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:42.979+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:43.295 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:43.290+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:43.420 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:43.414+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:43.649 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:43.644+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:43.674 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:43.669+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:43.738 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:43.722+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:43.826 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:43.821+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:43.942 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:43.937+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:44.282 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:44.277+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:44.470 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:44.465+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:44.636 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:44.631+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:44.681 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:44.676+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:44.682 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:44.678+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:44.816 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:44.810+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:44.951 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:44.947+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:45.266 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:45.261+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:45.459 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:45.453+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:45.653 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:45.645+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:45.682 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:45.676+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:45.686 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:45.681+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:45.799 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:45.794+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:45.974 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:45.969+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:46.250 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:46.246+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:46.465 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:46.460+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:46.647 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:46.642+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:46.660 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:46.655+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:46.720 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:46.714+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:46.837 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:46.832+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:47.030 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:47.024+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:47.236 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:47.228+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:47.493 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:47.489+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:47.640 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:47.635+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:47.669 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:47.664+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:47.760 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:47.756+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:47.852 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:47.848+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:48.076 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:48.070+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:48.256 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:48.250+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:48.498 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:48.494+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:48.627 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:48.621+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:48.698 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:48.694+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:48.797 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:48.792+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:48.855 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:48.850+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:49.037 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:49.031+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:49.285 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:49.280+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:49.508 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:49.503+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:49.596 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:49.590+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:49.715 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:49.708+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:49.827 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:49.821+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:49.828 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:49.823+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:50.042 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:50.036+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:50.307 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:50.301+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:50.561 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:50.556+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:50.649 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:50.640+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:50.727 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:50.722+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:50.861 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:50.855+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:50.864 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:50.857+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:51.024 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:51.018+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:51.273 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:51.267+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:51.567 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:51.561+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:51.604 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:51.599+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:51.718 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:51.712+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:51.885 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:51.880+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:51.894 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:51.889+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:52.056 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:52.051+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:52.235 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:52.230+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:52.559 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:52.554+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:52.596 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:52.589+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:52.758 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:52.753+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:52.844 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:52.839+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:52.872 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:52.868+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:53.063 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:53.059+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:53.283 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:53.278+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:53.607 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:53.602+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:53.638 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:53.634+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:53.724 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:53.719+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:53.837 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:53.833+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:53.870 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:53.864+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:54.031 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:54.026+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:54.258 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:54.253+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:54.603 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:54.598+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:54.631 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:54.626+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:54.778 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:54.773+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:54.827 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:54.822+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:54.900 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:54.895+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:55.074 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:55.069+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:55.314 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:55.308+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:55.601 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:55.596+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:55.612 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:55.607+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:55.814 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:55.809+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:55.838 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:55.833+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:55.935 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:55.930+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:56.040 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:56.035+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:56.307 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:56.302+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:56.561 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:56.556+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:56.596 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:56.592+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:56.849 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:56.844+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:56.865 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:56.861+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:56.986 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:56.981+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:57.094 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:57.089+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:57.336 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:57.331+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:57.558 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:57.553+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:57.611 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:57.606+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:57.852 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:57.847+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:57.871 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:57.867+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:58.029 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:58.024+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:58.091 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:58.087+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:58.322 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:58.318+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:58.520 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:58.512+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:58.655 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:58.651+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:58.859 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:58.853+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:58.913 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:58.909+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:06:59.080 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:06:59.077+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:06:59.106 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:06:59.102+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:06:59.341 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:06:59.337+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:06:59.553 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:06:59.548+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:06:59.711 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:06:59.706+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:06:59.811 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:06:59.807+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:06:59.939 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:06:59.935+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:00.038 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:00.033+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:00.136 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:00.132+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:00.344 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:00.338+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:00.536 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:00.531+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:00.704 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:00.700+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:00.859 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:00.854+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:00.940 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:00.934+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:01.045 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:01.041+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:01.161 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:01.155+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:01.375 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:01.370+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:01.512 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:01.506+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:01.738 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:01.733+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:01.896 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:01.891+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:01.901 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:01.895+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:02.087 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:02.082+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:02.143 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:02.139+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:02.369 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:02.363+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:02.501 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:02.497+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:02.705 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:02.698+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:02.852 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:02.847+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:02.935 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:02.930+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:03.069 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:03.064+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:03.140 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:03.135+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:03.409 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:03.402+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:03.544 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:03.540+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:03.688 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:03.682+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:03.837 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:03.832+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:03.899 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:03.894+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:04.101 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:04.096+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:04.136 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:04.131+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:04.408 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:04.403+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:04.522 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:04.517+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:04.712 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:04.707+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:04.851 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:04.845+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:04.913 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:04.907+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:05.094 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:05.089+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:05.128 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:05.122+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:05.369 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:05.364+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:05.488 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:05.483+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:05.761 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:05.755+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:05.872 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:05.866+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:05.880 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:05.874+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:06.084 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:06.080+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:06.087 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:06.082+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:06.347 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:06.342+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:06.515 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:06.511+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:06.780 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:06.774+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:06.876 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:06.870+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:06.917 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:06.912+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:07.124 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:07.118+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:07.127 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:07.122+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:07.346 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:07.341+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:07.534 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:07.529+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:07.746 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:07.741+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:07.887 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:07.880+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:07.959 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:07.954+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:08.109 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:08.106+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:08.153 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:08.149+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:08.335 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:08.329+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:08.529 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:08.524+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:08.720 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:08.714+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:08.905 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:08.900+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:08.955 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:08.949+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:09.121 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:09.117+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:09.155 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:09.151+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:09.301 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:09.296+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:09.580 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:09.576+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:09.692 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:09.686+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:09.869 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:09.863+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:09.990 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:09.984+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:10.094 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:10.089+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:10.196 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:10.192+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:10.316 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:10.311+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:10.623 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:10.618+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:10.744 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:10.738+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:10.854 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:10.849+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:11.022 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:11.017+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:11.093 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:11.088+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:11.221 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:11.217+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:11.324 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:11.319+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:11.350 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:07:11.344+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 42sec e285 v285) 2019-12-09T16:07:11.668 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:11.663+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:11.787 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:11.781+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:11.879 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:11.874+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:12.049 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:12.044+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:12.144 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:12.139+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:12.266 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:12.262+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:12.369 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:12.364+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:12.650 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:12.646+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:12.810 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:12.804+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:12.872 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:12.866+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:13.024 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:13.016+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:13.170 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:13.164+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:13.248 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:13.243+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:13.385 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:13.379+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:13.621 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:13.615+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:13.822 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:13.816+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:13.880 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:13.874+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:14.067 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:14.061+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:14.134 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:14.129+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:14.258 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:14.254+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:14.402 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:14.396+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:14.624 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:14.619+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:14.835 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:14.830+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:14.884 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:14.878+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:15.048 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:15.041+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:15.106 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:15.102+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:15.241 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:15.236+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:15.446 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:15.441+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:15.649 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:15.643+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:15.853 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:15.848+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:15.864 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:15.858+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:16.046 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:16.041+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:16.122 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:16.115+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:16.256 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:16.251+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:16.367 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:07:16.362+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 42sec e285 v285) 2019-12-09T16:07:16.420 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:16.415+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:16.632 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:16.626+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:16.884 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:16.878+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:16.901 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:16.895+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:17.038 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:17.033+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:17.098 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:17.094+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:17.248 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:17.243+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:17.427 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:17.423+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:17.604 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:17.599+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:17.866 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:17.860+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:17.904 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:17.898+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:18.090 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:18.085+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:18.112 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:18.107+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:18.278 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:18.273+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:18.477 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:18.472+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:18.616 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:18.611+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:18.911 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:18.905+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:18.945 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:18.939+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:19.076 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:19.070+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:19.143 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:19.136+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:19.258 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:19.254+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:19.514 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:19.509+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:19.596 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:19.590+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:19.871 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:19.865+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:19.978 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:19.973+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:20.089 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:20.085+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:20.142 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:20.137+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:20.251 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:20.246+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:20.482 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:20.476+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:20.560 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:20.555+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:20.826 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:07:20.820+0000 1694f700 -1 osd.5 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.288762+0000 front 2019-12-09T16:05:54.846743+0000 (oldest deadline 2019-12-09T16:06:38.217667+0000) 2019-12-09T16:07:21.014 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:07:21.008+0000 1694f700 -1 osd.7 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:55.022224+0000 front 2019-12-09T16:05:57.416517+0000 (oldest deadline 2019-12-09T16:06:36.404489+0000) 2019-12-09T16:07:21.128 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:21.123+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:21.193 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:07:21.188+0000 1694f700 -1 osd.4 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.246249+0000 front 2019-12-09T16:05:58.464540+0000 (oldest deadline 2019-12-09T16:06:36.298850+0000) 2019-12-09T16:07:21.295 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:07:21.291+0000 1694f700 -1 osd.1 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:52.100270+0000 front 2019-12-09T16:05:57.413113+0000 (oldest deadline 2019-12-09T16:06:36.539167+0000) 2019-12-09T16:07:21.441 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:07:21.435+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 42sec e285 v285) 2019-12-09T16:07:21.515 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:21.511+0000 1694f700 -1 osd.0 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.475215+0000 front 2019-12-09T16:05:56.574308+0000 (oldest deadline 2019-12-09T16:06:38.252000+0000) 2019-12-09T16:07:21.541 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:07:21.534+0000 1694f700 -1 osd.3 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:58.016717+0000 front 2019-12-09T16:05:55.029413+0000 (oldest deadline 2019-12-09T16:06:37.827632+0000) 2019-12-09T16:07:22.122 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:07:22.118+0000 1694f700 -1 osd.2 285 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:05:57.145009+0000 front 2019-12-09T16:05:57.437108+0000 (oldest deadline 2019-12-09T16:06:39.849437+0000) 2019-12-09T16:07:51.633 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:07:51.629+0000 1694f700 -1 osd.0 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116805 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 821797392 gen 100] snapc 0=[] ondisk+write+known_if_redirected e286) 2019-12-09T16:08:20.199 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:20.193+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:20.372 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:20.363+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:21.191 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:21.185+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:21.204 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:21.195+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:21.206 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:21.201+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:21.401 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:21.394+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:21.415 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:21.408+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:22.200 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:22.187+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:22.201 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:22.191+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:22.250 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:22.244+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:22.400 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:22.393+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:22.447 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:22.440+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:22.531 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:22.527+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:23.178 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:23.173+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:23.180 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:23.176+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:23.274 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:23.268+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:23.361 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:23.356+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:23.429 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:23.422+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:23.493 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:23.487+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:24.182 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:24.173+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:24.182 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:24.177+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:24.267 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:24.260+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:24.375 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:24.368+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:24.413 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:24.406+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:24.462 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:24.456+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:25.129 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:25.122+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:25.217 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:25.208+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:25.217 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:25.213+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:25.270 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:25.259+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:25.345 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:25.338+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:25.420 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:25.410+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:25.467 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:25.460+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:25.552 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:56:01.817 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:25.567 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:56:01.831 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:25.588 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:56:01.809 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:25.595 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:56:01.817 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:25.597 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:56:01.819 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:25.600 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:56:01.822 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:25.797 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:56:02.019 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:25.815 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:56:02.037 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:25.960 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:56:02.225 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:26.125 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:26.118+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:26.200 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:26.193+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:26.200 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:26.195+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:26.294 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:26.287+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:26.327 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:26.321+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:26.422 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:26.415+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:26.513 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:26.506+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:27.164 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:27.159+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:27.179 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:27.172+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:27.180 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:27.175+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:27.262 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:27.256+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:27.289 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:27.283+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:27.387 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:27.381+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:27.490 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:27.484+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:28.187 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:28.179+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:28.188 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:28.182+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:28.190 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:28.182+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:28.291 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:28.283+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:28.327 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:28.320+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:28.345 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:28.341+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:28.544 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:28.536+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:29.165 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:29.158+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:29.213 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:29.207+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:29.214 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:29.209+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:29.250 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:29.243+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:29.291 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:29.285+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:29.322 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:29.317+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:29.587 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:29.579+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:30.204 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:30.197+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:30.234 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:30.229+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:30.235 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:30.232+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:30.254 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:30.248+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:30.290 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:30.285+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:30.343 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:30.336+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:30.621 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:30.614+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:31.235 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:31.228+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:31.239 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:31.234+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:31.240 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:31.236+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:31.256 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:31.250+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:31.312 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:31.305+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:31.370 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:31.363+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:31.656 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:31.649+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:31.708 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:31.702+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:32.197 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:32.192+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:32.198 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:32.193+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:32.214 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:32.209+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:32.279 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:32.275+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:32.297 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:32.292+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:32.407 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:32.402+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:32.677 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:32.671+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:32.702 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:32.691+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 138 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:33.176 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:33.171+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:33.181 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:33.176+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:33.205 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:33.197+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:33.242 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:33.235+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:33.340 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:33.334+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:33.390 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:33.384+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:33.657 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:33.651+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:33.704 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:33.689+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:34.219 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:34.213+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:34.228 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:34.223+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:34.229 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:34.225+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:34.261 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:34.254+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:34.316 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:34.310+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:34.381 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:34.375+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:34.682 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:34.675+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:34.691 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:34.670+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:35.242 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:35.235+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:35.262 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:35.256+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:35.263 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:35.259+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:35.310 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:35.304+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:35.360 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:35.354+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:35.389 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:35.384+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:35.652 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:35.645+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:35.706 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:35.690+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:36.199 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:36.193+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:36.257 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:36.252+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:36.260 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:36.256+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:36.312 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:36.307+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:36.361 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:36.356+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:36.369 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:36.364+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:36.639 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:36.631+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:36.763 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:36.747+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:37.178 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:37.172+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:37.259 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:37.253+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:37.259 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:37.255+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:37.331 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:37.324+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:37.334 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:37.328+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:37.368 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:37.362+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:37.685 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:37.677+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:37.758 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:37.741+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:38.172 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:38.163+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:38.230 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:38.223+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:38.231 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:38.227+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:38.320 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:38.314+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:38.332 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:38.327+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:38.418 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:38.413+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:38.717 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:38.710+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:38.759 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:38.739+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:39.180 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:39.173+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:39.232 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:39.226+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:39.233 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:39.229+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:39.292 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:39.286+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:39.367 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:39.360+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:39.467 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:39.463+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:39.723 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:39.716+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:39.817 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:39.791+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:39.831 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:39.811+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:40.144 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:40.137+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:40.235 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:40.227+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:40.239 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:40.235+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:40.270 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:40.265+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:40.357 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:40.350+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:40.510 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:40.504+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:40.702 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:40.695+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:40.857 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:40.831+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:40.872 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:40.852+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:41.111 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:41.103+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:41.288 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:41.283+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:41.289 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:41.285+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:41.308 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:41.303+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:41.344 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:41.337+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:41.564 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:41.558+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:41.747 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:41.740+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:41.876 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:41.851+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:41.894 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:41.871+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:42.123 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:42.116+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:42.336 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:42.329+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:42.336 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:42.330+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:42.337 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:42.332+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:42.387 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:42.380+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:42.560 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:42.554+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:42.725 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:42.718+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:42.925 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:42.901+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:42.939 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:42.921+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:43.103 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:43.096+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:43.360 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:43.355+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:43.363 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:43.359+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:43.367 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:43.362+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:43.401 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:43.394+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:43.544 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:43.537+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:43.760 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:43.752+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:43.996 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:43.965+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:44.011 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:43.987+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:44.134 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:44.127+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:44.335 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:44.331+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:44.344 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:44.340+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:44.346 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:44.343+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:44.382 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:44.376+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:44.500 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:44.495+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:44.777 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:44.770+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:44.974 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:44.950+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:44.987 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:44.969+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:45.157 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:45.150+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:45.346 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:45.342+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:45.348 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:45.344+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:45.370 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:45.363+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:45.372 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:45.367+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:45.467 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:45.463+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:45.785 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:45.778+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:45.952 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:45.946+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:45.983 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:45.966+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:46.181 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:46.174+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:46.337 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:46.329+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:46.391 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:46.384+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:46.392 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:46.387+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:46.423 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:46.419+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:46.449 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:46.445+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:46.825 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:46.818+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:46.970 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:46.964+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:46.997 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:46.979+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:47.145 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:47.137+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:47.386 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:47.379+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:47.387 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:47.383+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:47.388 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:47.381+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:47.441 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:47.438+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:47.470 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:47.465+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:47.848 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:47.840+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:47.993 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:47.974+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:48.008 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:47.989+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:48.132 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:48.126+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:48.403 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:48.396+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:48.427 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:48.421+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:48.438 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:48.431+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:48.439 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:48.434+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:48.476 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:48.471+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:48.836 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:48.829+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:48.987 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:48.962+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:49.004 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:48.983+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:49.090 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:08:49.083+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:36.411013+0000 front 2019-12-09T16:07:36.420012+0000 (oldest deadline 2019-12-09T16:08:19.943982+0000) 2019-12-09T16:08:49.412 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:49.405+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:49.423 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:49.417+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:49.424 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:49.420+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:49.430 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:49.424+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:49.446 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:49.441+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:49.888 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:49.882+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:49.965 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:49.949+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:49.982 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:49.961+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:50.410 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:50.402+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:50.411 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:50.406+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:50.466 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:50.458+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:50.467 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:50.461+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:50.477 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:50.473+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:50.897 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:50.889+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:50.996 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:50.981+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:51.010 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:50.992+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:51.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:51.392+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:51.398 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:51.394+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:51.482 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:51.474+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:51.511 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:51.506+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:51.524 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:51.519+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:51.924 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:51.916+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:51.927 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:08:51.917+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 41sec e292 v292) 2019-12-09T16:08:52.060 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:52.043+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:52.066 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:52.055+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:52.398 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:52.393+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:52.398 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:52.394+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:52.482 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:52.477+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:52.509 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:52.502+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:52.560 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:52.554+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:52.945 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:52.937+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:53.032 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:53.008+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:53.045 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:53.027+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:53.412 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:53.406+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:53.413 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:53.409+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:53.443 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:53.439+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:53.537 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:53.532+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:53.562 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:53.555+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:53.936 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:53.929+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:54.065 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:54.042+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:54.080 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:54.061+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:54.368 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:54.363+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:54.370 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:54.365+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:54.443 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:54.437+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:54.511 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:54.507+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:54.575 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:54.568+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:54.909 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:54.901+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:55.119 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:55.094+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:55.133 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:55.114+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:55.417 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:55.409+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:55.418 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:55.414+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:55.492 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:55.486+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:55.512 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:55.507+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:55.613 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:55.607+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:55.931 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:55.924+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:56.138 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:56.132+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:56.167 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:56.148+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:56.407 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:56.400+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:56.408 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:56.403+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:56.537 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:56.531+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:56.547 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:56.541+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:56.631 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:56.624+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:56.953 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:08:56.943+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 41sec e292 v292) 2019-12-09T16:08:56.975 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:56.967+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:57.129 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:57.110+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:57.149 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:57.131+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:57.385 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:57.380+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:57.387 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:57.382+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:57.551 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:57.545+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:57.587 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:57.582+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:57.681 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:57.674+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:58.000 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:57.993+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:58.158 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:58.134+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:58.172 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:58.153+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:58.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:58.392+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:58.400 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:58.396+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:58.550 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:58.545+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:58.636 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:58.630+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:58.707 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:58.699+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:08:58.989 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:08:58.983+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:08:59.226 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:59.203+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:08:59.239 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:08:59.222+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:08:59.362 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:59.356+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:08:59.362 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:08:59.358+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:08:59.603 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:08:59.598+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:08:59.627 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:08:59.622+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:08:59.673 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:08:59.666+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:00.012 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:00.004+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:00.212 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:00.187+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:00.220 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:00.207+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:00.340 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:00.334+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:00.341 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:00.337+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:00.594 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:00.588+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:00.604 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:00.599+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:00.666 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:00.658+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:01.031 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:01.024+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:01.215 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:01.190+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:01.228 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:01.210+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:01.377 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:01.373+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:01.379 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:01.376+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:01.605 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:01.600+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:01.606 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:01.600+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:01.644 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:01.636+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:02.047 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:02.038+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:02.054 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:09:02.048+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 41sec e292 v292) 2019-12-09T16:09:02.274 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:02.260+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:02.280 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:02.269+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:02.366 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:02.359+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:02.368 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:02.363+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:02.604 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:02.599+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:02.609 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:02.602+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:02.635 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:02.629+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:03.101 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:03.094+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:03.281 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:03.257+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:03.295 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:03.276+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:03.394 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:03.387+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:03.395 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:03.390+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:03.627 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:03.621+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:03.646 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:03.639+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:03.677 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:03.670+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:04.092 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:04.086+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:04.234 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:04.229+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:04.252 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:04.241+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:04.433 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:04.428+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:04.437 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:04.432+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:04.614 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:04.609+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:04.673 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:04.668+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:04.689 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:04.682+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:05.127 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:05.117+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:05.271 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:05.265+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:05.301 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:05.286+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:05.440 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:05.434+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:05.443 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:05.439+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:05.658 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:05.652+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:05.686 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:05.680+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:05.717 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:05.710+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:06.102 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:06.095+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:06.309 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:06.303+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:06.337 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:06.324+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:06.404 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:06.398+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:06.408 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:06.404+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:06.670 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:06.665+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:06.692 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:06.685+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:06.728 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:06.723+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:07.080 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:09:07.071+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 41sec e292 v292) 2019-12-09T16:09:07.120 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:07.113+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:07.337 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:07.314+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:07.344 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:07.333+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:07.392 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:07.387+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:07.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:07.393+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:07.666 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:07.659+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:07.681 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:07.674+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:07.776 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:07.771+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:08.074 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:08.068+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:08.312 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:08.304+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:08.341 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:08.326+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:08.436 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:08.430+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:08.439 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:08.435+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:08.685 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:08.679+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:08.687 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:08.683+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:08.781 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:08.776+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:09.086 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:09.079+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:09.315 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:09.293+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:09.327 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:09.310+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:09.483 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:09.477+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:09.485 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:09.480+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:09.685 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:09.680+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:09.692 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:09.685+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:09.831 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:09.825+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:10.111 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:10.105+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:10.349 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:10.326+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:10.358 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:10.345+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:10.529 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:10.522+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:10.534 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:10.530+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:10.712 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:10.704+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:10.723 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:10.717+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:10.834 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:10.829+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:11.074 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:11.067+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:11.395 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:11.372+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:11.409 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:11.390+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:11.565 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:11.560+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:11.567 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:11.564+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:11.703 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:11.696+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:11.738 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:11.732+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:11.820 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:11.815+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:12.093 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:12.086+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:12.105 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:09:12.100+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 41sec e292 v292) 2019-12-09T16:09:12.451 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:12.431+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:12.462 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:12.447+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:12.521 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:12.517+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:12.522 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:12.518+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:12.692 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:12.684+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:12.710 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:12.705+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:12.775 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:12.769+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:13.066 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:13.059+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:13.449 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:13.428+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:13.463 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:13.445+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:13.495 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:13.486+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:13.496 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:13.491+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:13.733 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:13.726+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:13.757 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:13.751+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:13.824 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:13.817+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:14.104 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:14.098+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:14.452 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:14.427+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:14.466 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:14.447+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:14.503 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:14.494+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:14.505 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:14.499+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:14.706 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:14.699+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:14.777 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:14.770+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:14.789 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:14.783+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:15.133 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:15.126+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:15.474 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:15.467+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:15.478 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:15.473+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:15.495 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:15.489+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:15.528 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:15.511+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:15.703 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:15.696+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:15.776 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:15.770+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:15.781 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:15.775+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:16.113 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:16.106+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:16.486 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:16.477+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.848654+0000 front 2019-12-09T16:07:38.605083+0000 (oldest deadline 2019-12-09T16:08:20.880556+0000) 2019-12-09T16:09:16.488 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:16.481+0000 1694f700 -1 osd.0 292 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:16.571 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:16.546+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:07:58.738074+0000 (oldest deadline 2019-12-09T16:08:38.738074+0000) 2019-12-09T16:09:16.584 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:16.567+0000 1694f700 -1 osd.6 292 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:16.666 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:09:16.658+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.705960+0000 front 2019-12-09T16:07:38.833850+0000 (oldest deadline 2019-12-09T16:08:24.534228+0000) 2019-12-09T16:09:16.787 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:16.781+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:35.876912+0000 front 2019-12-09T16:07:36.410609+0000 (oldest deadline 2019-12-09T16:08:21.104781+0000) 2019-12-09T16:09:16.812 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:09:16.807+0000 1694f700 -1 osd.3 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:37.919708+0000 front 2019-12-09T16:07:38.179624+0000 (oldest deadline 2019-12-09T16:08:22.445871+0000) 2019-12-09T16:09:17.169 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:09:17.162+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:07:38.605421+0000 front 2019-12-09T16:07:38.848880+0000 (oldest deadline 2019-12-09T16:08:20.283483+0000) 2019-12-09T16:09:17.183 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:09:17.178+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 41sec e292 v292) 2019-12-09T16:09:17.829 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:09:17.743+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 163 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e289) 2019-12-09T16:09:17.870 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:17.645+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:18.952 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:09:18.797+0000 1694f700 -1 osd.0 294 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:116848 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-09T16:09:20.119 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:09:20.112+0000 20e25700 -1 RGWWatcher::handle_error cookie 398400336 err (107) Transport endpoint is not connected 2019-12-09T16:09:45.244 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:09:45.027+0000 1694f700 -1 osd.1 293 get_health_metrics reporting 1 slow ops, oldest is osd_repop(osd.0.0:29285 5.5 e292/284 5:a93a5511:::notify.2:head v 292'225) 2019-12-09T16:09:48.527 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:48.492+0000 34a75700 -1 osd.2 296 build_incremental_map_msg missing incremental map 296 2019-12-09T16:09:48.714 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:09:48.652+0000 37a7b700 -1 osd.2 296 build_incremental_map_msg missing incremental map 296 2019-12-09T16:10:47.256 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:47.252+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:47.709 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:47.704+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:48.272 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:48.266+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:48.717 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:48.713+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:49.296 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:49.291+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:49.668 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:10:49.663+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:10:49.706 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:49.701+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:50.262 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:50.256+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:50.267 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:10:50.262+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:10:50.623 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:10:50.619+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:10:50.750 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:50.743+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:50.985 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:10:50.980+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:10:51.231 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:51.225+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:51.298 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:10:51.292+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:10:51.667 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:10:51.661+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:10:51.771 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:51.763+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:51.991 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:10:51.984+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:10:51.995 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:10:51.989+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:10:52.030 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:10:52.025+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:10:52.228 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:52.221+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:52.315 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:10:52.305+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:10:52.653 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:10:52.648+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:10:52.801 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:52.796+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:53.001 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:10:52.995+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:10:53.019 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:10:53.010+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:10:53.079 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:10:53.071+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:10:53.244 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:53.238+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:53.350 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:10:53.345+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:10:53.679 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:10:53.673+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:10:53.812 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:53.808+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:54.006 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:10:54.000+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:10:54.011 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:10:54.006+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:10:54.108 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:10:54.102+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:10:54.216 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:54.211+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:54.370 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:10:54.365+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:10:54.684 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:10:54.679+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:10:54.859 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:54.854+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:54.980 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:10:54.974+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:10:54.995 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:10:54.990+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:10:55.149 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:10:55.144+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:10:55.209 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:55.202+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:55.351 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:10:55.347+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:10:55.679 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:10:55.674+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:10:55.826 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:55.821+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:56.001 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:10:55.994+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:10:56.010 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:10:56.005+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:10:56.137 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:10:56.132+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:10:56.262 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:56.257+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:56.330 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:10:56.326+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:10:56.706 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:10:56.700+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:10:56.832 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:56.826+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:56.981 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:10:56.977+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:10:57.022 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:10:57.017+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:10:57.125 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:10:57.120+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:10:57.295 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:57.290+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:57.357 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:10:57.353+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:10:57.693 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:10:57.686+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:10:57.858 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:57.854+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:57.988 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:10:57.982+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:10:57.990 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:10:57.986+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:10:58.096 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:10:58.090+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:10:58.291 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:58.285+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:58.316 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:10:58.310+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:10:58.740 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:10:58.735+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:10:58.870 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:58.866+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:10:58.991 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:10:58.984+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:10:59.042 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:10:59.037+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:10:59.114 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:10:59.107+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:10:59.301 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:10:59.297+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:10:59.314 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:10:59.309+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:10:59.724 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:10:59.718+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:10:59.871 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:10:59.866+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:00.015 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:00.010+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:00.026 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:00.022+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:00.165 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:00.160+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:00.349 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:00.344+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:00.361 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:00.355+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:00.703 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:00.697+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:00.860 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:00.855+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:01.009 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:01.005+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:01.049 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:01.044+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:01.183 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:01.177+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:01.330 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:01.325+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:01.338 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:01.333+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:01.723 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:01.717+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:01.835 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:01.830+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:01.969 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:01.963+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:02.015 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:02.008+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:02.142 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:02.136+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:02.338 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:02.333+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:02.365 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:02.361+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:02.711 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:02.705+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:02.796 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:02.791+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:02.942 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:02.938+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:03.007 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:03.002+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:03.128 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:03.121+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:03.357 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:03.352+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:03.391 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:03.385+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:03.708 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:03.700+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:03.799 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:03.794+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:03.970 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:03.966+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:03.977 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:03.972+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:04.108 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:04.102+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:04.365 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:04.360+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:04.416 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:04.411+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:04.723 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:04.716+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:04.779 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:04.774+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:04.946 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:04.941+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:04.961 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:04.957+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:05.149 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:05.142+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:05.355 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:05.350+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:05.412 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:05.406+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:05.740 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:05.734+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:05.772 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:05.768+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:05.967 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:05.962+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:05.977 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:05.968+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:06.119 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:06.113+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:06.379 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:06.373+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:06.391 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:06.387+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:06.735 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:06.729+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:06.739 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:06.734+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:06.935 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:06.930+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:06.977 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:06.972+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:07.114 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:07.108+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:07.361 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:07.356+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:07.383 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:07.378+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:07.689 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:07.684+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:07.785 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:07.768+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:07.907 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:07.902+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:07.961 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:07.957+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:08.084 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:08.078+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:08.338 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:08.334+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:08.391 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:08.384+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:08.687 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:08.681+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:08.761 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:08.756+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:08.946 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:08.942+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:09.002 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:08.997+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:09.136 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:09.131+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:09.349 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:09.345+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:09.414 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:09.408+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:09.684 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:09.678+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:09.788 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:09.784+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:09.904 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:09.900+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:10.041 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:10.036+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:10.182 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:10.176+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:10.361 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:10.356+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:10.383 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:10.377+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:10.662 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:10.657+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:10.809 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:10.805+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:10.898 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:10.893+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:11.049 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:11.038+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:11.166 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:11.160+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:11.339 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:11.334+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:11.339 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:11.334+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:11.667 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:11.662+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:11.842 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:11.838+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:11.908 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:11.903+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:12.062 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:12.057+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:12.167 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:12.161+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:12.313 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:12.308+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:12.378 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:12.372+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:12.634 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:12.629+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:12.877 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:12.872+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:12.935 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:12.931+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:13.029 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:13.024+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:13.215 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:13.209+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:13.328 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:13.323+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:13.426 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:13.420+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:13.595 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:13.588+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:13.835 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:13.831+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:13.971 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:13.966+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:14.041 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:14.035+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:14.178 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:14.172+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:14.346 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:14.341+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:14.458 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:14.452+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:14.549 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:14.543+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:14.849 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:14.844+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:14.961 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:14.956+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:15.040 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:15.036+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:15.173 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:15.167+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:15.340 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:15.335+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:15.493 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:15.487+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:15.513 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:15.501+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:15.818 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:15.812+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:15.993 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:15.988+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:16.040 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:16.036+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:16.197 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:16.192+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:16.296 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:16.292+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:16.447 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:16.442+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:16.519 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:16.513+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:16.815 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:16.810+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:17.004 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:16.999+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:17.009 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:17.004+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:17.247 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:17.241+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:17.351 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:17.346+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:17.435 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:17.430+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:17.498 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:17.493+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:17.816 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:17.812+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:17.987 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:17.982+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:18.045 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:18.040+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:18.210 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:18.204+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:18.329 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:18.324+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:18.450 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:18.444+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:18.491 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:18.485+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:18.786 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:18.781+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:19.012 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:19.008+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:19.092 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:19.086+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:19.201 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:19.195+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:19.334 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:19.329+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:19.433 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:19.428+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:19.477 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:19.471+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:19.825 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:19.820+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:19.976 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:19.971+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:20.124 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:20.119+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:20.170 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:20.165+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:20.292 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:20.287+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:20.453 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:20.446+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:20.504 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:20.498+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:20.780 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:20.775+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:20.990 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:20.985+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:21.104 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:21.099+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:21.167 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:21.161+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:21.310 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:21.306+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:21.441 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:21.436+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:21.488 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:21.482+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:21.778 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:21.772+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:22.009 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:22.003+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:22.109 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:22.104+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:22.137 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:22.132+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:22.360 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:22.355+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:22.467 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:22.462+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:22.504 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:22.498+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:22.642 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:11:22.636+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 43sec e298 v298) 2019-12-09T16:11:22.777 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:22.772+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:23.019 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:23.014+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:23.136 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:23.130+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:23.161 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:23.157+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:23.400 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:23.394+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:23.476 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:23.470+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:23.523 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:23.517+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:23.798 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:23.793+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:24.023 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:24.018+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:24.150 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:24.144+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:24.154 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:24.148+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:24.377 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:24.372+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:24.488 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:24.483+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:24.558 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:24.553+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:24.838 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:24.833+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:25.041 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:25.036+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:25.162 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:25.157+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:25.197 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:25.187+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:25.333 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:25.329+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:25.452 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:25.445+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:25.605 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:25.599+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:25.841 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:25.837+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:26.103 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:26.090+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:26.170 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:26.165+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:26.172 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:26.168+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:26.359 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:26.355+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:26.438 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:26.431+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:26.576 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:26.569+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:26.892 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:26.886+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:27.145 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:27.140+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:27.163 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:27.158+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:27.223 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:27.217+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:27.324 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:27.315+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:27.492 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:27.486+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:27.554 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:27.546+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:27.658 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:11:27.651+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 43sec e298 v298) 2019-12-09T16:11:27.920 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:27.915+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:28.122 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:28.116+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:28.188 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:28.183+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:28.202 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:28.196+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:28.274 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:28.270+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:28.473 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:28.468+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:28.522 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:28.516+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:28.947 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:28.942+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:29.099 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:29.094+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:29.161 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:29.155+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:29.244 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:29.238+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:29.271 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:29.267+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:29.501 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:29.494+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:29.528 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:29.522+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:29.966 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:29.960+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:30.112 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:30.107+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:30.135 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:30.131+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:30.222 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:30.217+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:30.282 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:30.278+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:30.493 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:30.488+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:30.530 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:30.525+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:30.999 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:30.994+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:31.157 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:31.151+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:31.159 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:31.155+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:31.220 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:31.214+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:31.300 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:31.295+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:31.484 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:31.478+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:31.554 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:31.548+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:32.034 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:11:32.030+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793992+0000 front 2019-12-09T16:10:07.129434+0000 (oldest deadline 2019-12-09T16:10:46.944516+0000) 2019-12-09T16:11:32.142 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:11:32.138+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119619+0000 front 2019-12-09T16:10:06.507831+0000 (oldest deadline 2019-12-09T16:10:50.407221+0000) 2019-12-09T16:11:32.154 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:32.150+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:32.187 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:11:32.182+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.199212+0000 front 2019-12-09T16:10:07.206780+0000 (oldest deadline 2019-12-09T16:10:51.807227+0000) 2019-12-09T16:11:32.256 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:11:32.251+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.793920+0000 front 2019-12-09T16:10:07.787485+0000 (oldest deadline 2019-12-09T16:10:49.766354+0000) 2019-12-09T16:11:32.521 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:11:32.514+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:04.268111+0000 front 2019-12-09T16:10:09.375390+0000 (oldest deadline 2019-12-09T16:10:48.875706+0000) 2019-12-09T16:11:32.535 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:11:32.529+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:07.463747+0000 front 2019-12-09T16:10:06.631306+0000 (oldest deadline 2019-12-09T16:10:46.566329+0000) 2019-12-09T16:11:32.709 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:11:32.704+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 43sec e298 v298) 2019-12-09T16:11:33.172 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:11:33.167+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:10:09.119211+0000 front 2019-12-09T16:10:09.107758+0000 (oldest deadline 2019-12-09T16:10:51.405713+0000) 2019-12-09T16:11:57.959 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:59:34.223 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:57.966 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:59:34.231 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:58.045 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:59:34.267 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:58.052 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:59:34.275 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:58.058 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:59:34.280 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:58.062 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:59:34.284 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:58.299 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:59:34.564 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:58.304 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:59:34.526 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:58.305 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:59:34.527 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:03.761 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:03.756+0000 1694f700 -1 osd.7 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:117201 5.9 5:93e5b521:::notify.7:head [watch reconnect cookie 821997200 gen 88] snapc 0=[] ondisk+write+known_if_redirected e299) 2019-12-09T16:12:20.507 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:59:56.771 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:20.526 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:59:56.790 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:20.535 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:59:56.757 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:20.538 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:59:56.761 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:20.559 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:59:56.781 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:20.565 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:59:56.787 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:20.630 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:06:59:56.852 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:20.645 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:06:59:56.910 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:20.656 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:06:59:56.878 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:31.430 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:31.426+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:32.459 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:32.454+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:32.883 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:32.876+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:33.187 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:33.178+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:33.275 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:33.267+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:33.404 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:33.398+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:33.407 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:33.404+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:33.475 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:33.469+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:33.816 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:33.810+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:33.885 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:33.876+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:34.157 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:34.151+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:34.273 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:34.268+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:34.390 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:34.386+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:34.392 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:34.388+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:34.525 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:34.518+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:34.825 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:34.818+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:34.900 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:34.893+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:35.197 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:35.189+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:35.306 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:35.299+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:35.415 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:35.408+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:35.416 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:35.411+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:35.533 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:35.524+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:35.844 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:35.837+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:35.898 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:35.891+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:36.242 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:36.234+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:36.272 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:36.266+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:36.379 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:36.372+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:36.381 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:36.376+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:36.535 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:36.530+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:36.831 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:36.825+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:36.935 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:36.928+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:37.240 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:37.233+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:37.277 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:37.270+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:37.346 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:37.340+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:37.347 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:37.342+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:37.552 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:37.547+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:37.835 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:37.828+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:37.945 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:37.938+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:38.288 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:38.282+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:38.291 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:38.283+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:38.374 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:38.368+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:38.374 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:38.371+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:38.515 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:38.505+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:38.874 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:38.868+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:38.934 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:38.927+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:39.280 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:39.274+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:39.300 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:39.293+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:39.338 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:39.332+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:39.339 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:39.335+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:39.471 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:39.466+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:39.907 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:39.902+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:39.965 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:39.958+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:40.293 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:40.286+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:40.317 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:40.311+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:40.355 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:40.348+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:40.356 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:40.352+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:40.510 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:40.504+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:40.865 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:40.860+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:41.015 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:41.006+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:41.289 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:41.281+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:41.316 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:41.310+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:41.340 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:41.335+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:41.344 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:41.340+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:41.483 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:41.478+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:41.821 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:41.816+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:42.062 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:42.055+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:42.331 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:42.324+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:42.351 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:42.345+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:42.355 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:42.351+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:42.361 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:42.354+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:42.455 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:42.449+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:42.779 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:42.774+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:43.092 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:43.085+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:43.330 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:43.323+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:43.330 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:43.326+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:43.331 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:43.318+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:43.388 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:43.381+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:43.460 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:43.455+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:43.573 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:43.567+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:43.749 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:43.734+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:44.114 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:44.106+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:44.299 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:44.292+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:44.309 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:44.303+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:44.310 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:44.306+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:44.433 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:44.427+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:44.438 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:44.431+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:44.616 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:44.600+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 103 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:44.700 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:44.694+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:45.095 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:45.088+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:45.268 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:45.260+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:45.269 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:45.263+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:45.297 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:45.291+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:45.389 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:45.382+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:45.483 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:45.476+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:45.603 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:45.590+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:45.678 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:45.673+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:46.110 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:46.103+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:46.222 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:46.216+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:46.223 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:46.219+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:46.312 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:46.305+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:46.416 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:46.412+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:46.522 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:46.517+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:46.627 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:46.609+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:46.680 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:46.672+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:47.072 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:47.065+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:47.237 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:47.232+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:47.238 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:47.235+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:47.361 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:47.352+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:47.444 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:47.438+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:47.499 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:47.491+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:47.610 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:47.589+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:47.653 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:47.648+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:48.077 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:48.071+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:48.193 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:48.189+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:48.195 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:48.191+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:48.317 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:48.311+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:48.401 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:48.396+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:48.468 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:48.461+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:48.566 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:48.552+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:48.688 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:48.680+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:49.070 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:49.063+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:49.233 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:49.227+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:49.233 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:49.229+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:49.302 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:49.295+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:49.412 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:49.406+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:49.471 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:49.465+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:49.601 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:49.576+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:12:49.614 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:49.596+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:49.696 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:49.689+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:50.077 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:50.070+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:50.223 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:50.218+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:50.225 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:50.219+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:50.263 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:50.255+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:50.375 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:50.368+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:50.491 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:50.484+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:50.580 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:50.574+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:12:50.608 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:50.592+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:50.735 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:50.729+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:51.126 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:51.119+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:51.255 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:51.249+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:51.256 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:51.251+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:51.287 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:51.281+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:51.331 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:51.325+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:51.487 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:51.480+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:51.658 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:51.634+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:12:51.672 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:51.653+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:51.708 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:51.703+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:52.128 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:52.119+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:52.258 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:52.253+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:52.259 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:52.255+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:52.327 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:52.319+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:52.356 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:52.352+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:52.497 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:52.491+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:52.701 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:52.677+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:12:52.712 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:52.697+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:52.738 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:52.732+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:53.106 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:53.099+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:53.257 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:53.252+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:53.260 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:53.256+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:53.281 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:53.274+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:53.330 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:53.323+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:53.520 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:53.513+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:53.724 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:53.699+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:12:53.735 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:53.717+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:53.747 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:53.742+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:54.082 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:54.075+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:54.310 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:54.305+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:54.312 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:54.307+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:54.334 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:54.327+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:54.340 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:54.335+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:54.516 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:54.511+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:54.735 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:54.729+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:54.737 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:54.713+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:12:54.751 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:54.733+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:55.123 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:55.116+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:55.271 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:55.260+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:55.272 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:55.267+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:55.313 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:55.307+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:55.368 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:55.363+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:55.526 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:55.519+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:55.713 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:55.701+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:55.734 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:55.705+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:12:55.754 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:55.730+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:56.158 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:56.150+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:56.276 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:56.269+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:56.306 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:56.302+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:56.309 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:56.305+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:56.407 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:56.399+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:56.535 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:56.528+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:56.692 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:56.686+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:56.785 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:56.779+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:12:56.816 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:56.801+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:57.138 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:57.131+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:57.311 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:57.304+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:57.322 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:57.316+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:57.323 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:57.318+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:57.443 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:57.437+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:57.553 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:57.546+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:57.742 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:57.736+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:57.785 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:57.762+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:12:57.797 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:57.780+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:58.158 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:58.151+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:58.313 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:58.307+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:58.329 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:58.323+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:58.333 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:58.329+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:58.416 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:58.410+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:58.539 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:58.533+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:58.797 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:58.791+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:58.819 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:58.797+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:12:58.832 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:58.813+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:12:59.119 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:12:59.112+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:12:59.297 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:12:59.289+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:12:59.319 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:59.312+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:12:59.320 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:12:59.315+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:12:59.435 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:12:59.429+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:12:59.575 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:12:59.568+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:12:59.774 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:12:59.767+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:12:59.840 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:59.817+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:12:59.853 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:12:59.833+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:00.151 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:00.144+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:00.311 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:00.304+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:00.312 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:00.308+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:00.319 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:00.311+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:00.413 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:00.409+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:00.584 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:00.577+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:00.754 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:00.748+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:00.902 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:00.879+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:00.916 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:00.897+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:01.168 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:01.161+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:01.318 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:01.311+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:01.320 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:01.315+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:01.355 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:01.348+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:01.424 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:01.420+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:01.566 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:01.560+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:01.725 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:01.719+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:01.952 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:01.929+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:01.966 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:01.947+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:02.181 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:02.175+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:02.369 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:02.364+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:02.369 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:02.365+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:02.384 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:02.378+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:02.470 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:02.466+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:02.551 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:02.544+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:02.770 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:02.765+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:02.938 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:02.931+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:02.969 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:02.950+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:03.191 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:03.185+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:03.269 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:13:03.261+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e305 v305) 2019-12-09T16:13:03.347 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:03.343+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:03.348 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:03.344+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:03.382 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:03.373+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:03.471 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:03.467+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:03.518 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:03.509+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:03.806 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:03.800+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:03.930 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:03.906+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:03.944 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:03.925+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:04.211 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:04.202+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:04.383 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:04.376+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:04.389 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:04.383+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:04.390 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:04.385+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:04.515 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:04.511+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:04.545 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:04.538+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:04.786 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:04.780+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:04.936 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:04.911+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:04.950 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:04.931+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:05.261 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:05.254+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:05.362 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:05.356+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:05.364 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:05.359+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:05.439 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:05.432+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:05.500 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:05.494+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:05.561 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:05.555+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:05.764 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:05.758+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:05.957 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:05.932+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:05.970 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:05.951+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:06.295 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:06.288+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:06.365 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:06.359+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:06.370 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:06.366+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:06.399 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:06.393+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:06.534 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:06.527+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:06.592 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:06.587+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:06.791 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:06.780+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:06.985 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:06.978+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:07.017 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:06.999+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:07.296 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:07.288+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:07.361 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:07.354+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:07.401 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:07.393+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:07.413 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:07.397+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:07.569 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:07.563+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:07.610 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:07.604+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:07.772 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:07.768+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:08.053 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:08.032+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:08.068 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:08.050+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:08.250 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:08.245+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:08.302 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:13:08.287+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e305 v305) 2019-12-09T16:13:08.400 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:08.394+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:08.421 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:08.416+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:08.423 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:08.417+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:08.586 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:08.579+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:08.593 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:08.589+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:08.752 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:08.747+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:09.084 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:09.070+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:09.110 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:09.092+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:09.222 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:09.216+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:09.387 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:09.379+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:09.388 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:09.383+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:09.404 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:09.398+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:09.612 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:09.606+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:09.623 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:09.618+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:09.708 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:09.702+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:10.155 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:10.138+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:10.174 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:10.151+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:10.222 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:10.217+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:10.374 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:10.368+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:10.375 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:10.371+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:10.400 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:10.392+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:10.603 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:10.595+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:10.620 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:10.614+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:10.685 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:10.680+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:11.168 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:11.147+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:11.182 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:11.165+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:11.193 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:11.187+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:11.345 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:11.338+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:11.345 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:11.341+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:11.412 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:11.406+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:11.605 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:11.598+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:11.620 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:11.614+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:11.733 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:11.727+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:12.177 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:12.155+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:12.189 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:12.173+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:12.217 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:12.212+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:12.379 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:12.374+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:12.384 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:12.380+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:12.387 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:12.382+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:12.624 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:12.618+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:12.643 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:12.638+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:12.756 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:12.752+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:13.161 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:13.155+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:13.194 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:13.177+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:13.220 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:13.215+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:13.323 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:13:13.313+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e305 v305) 2019-12-09T16:13:13.357 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:13.352+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:13.438 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:13.430+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:13.439 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:13.434+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:13.610 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:13.603+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:13.685 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:13.677+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:13.744 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:13.740+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:14.148 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:14.143+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:14.175 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:14.161+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:14.249 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:14.243+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:14.389 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:14.382+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:14.457 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:14.452+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:14.463 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:14.458+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:14.630 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:14.624+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:14.736 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:14.729+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:14.780 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:14.775+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:15.126 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:15.121+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:15.160 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:15.142+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:15.217 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:15.211+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:15.365 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:15.358+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:15.426 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:15.420+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:15.430 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:15.424+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:15.642 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:15.636+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:15.767 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:15.761+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:15.772 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:15.761+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:16.187 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:16.181+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:16.221 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:16.205+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:16.223 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:16.218+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:16.335 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:16.328+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:16.439 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:16.433+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:16.440 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:16.435+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:16.654 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:16.648+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:16.811 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:16.804+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:16.811 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:16.805+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:17.216 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:17.198+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:17.221 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:17.215+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:17.228 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:17.213+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:17.382 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:17.376+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:17.476 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:17.467+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:17.477 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:17.469+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:17.631 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:17.626+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:17.777 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:17.770+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:17.852 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:17.842+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:18.236 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:18.222+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:18.244 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:18.233+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:18.261 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:18.257+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:18.358 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:13:18.350+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e305 v305) 2019-12-09T16:13:18.395 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:18.390+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:18.469 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:18.463+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:18.469 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:18.465+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:18.629 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:18.623+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:18.806 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:18.799+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:18.819 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:18.812+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:19.252 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:19.240+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:19.288 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:19.283+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:19.315 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:19.300+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:19.417 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:19.412+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:19.515 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:19.509+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:19.523 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:19.512+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:19.659 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:19.653+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:19.803 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:19.797+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:19.816 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:19.810+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:20.226 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:20.220+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:20.335 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:20.313+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:20.352 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:20.332+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:20.462 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:20.457+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:20.525 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:20.520+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:20.532 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:20.526+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:20.668 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:20.662+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:20.789 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:20.783+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:20.838 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:20.832+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:21.205 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:21.199+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:21.388 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:21.383+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:21.413 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:21.399+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:21.469 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:21.463+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:21.533 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:21.528+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:21.534 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:21.530+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:21.693 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:21.686+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:21.812 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:21.807+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:21.854 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:21.848+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:22.249 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:22.245+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:22.391 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:22.369+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:22.405 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:22.387+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:22.440 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:22.434+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:22.531 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:22.521+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:22.534 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:22.528+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:22.711 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:22.706+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:22.829 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:13:22.822+0000 1694f700 -1 osd.3 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:47.266264+0000 front 2019-12-09T16:11:48.345114+0000 (oldest deadline 2019-12-09T16:12:31.316166+0000) 2019-12-09T16:13:22.843 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:13:22.837+0000 1694f700 -1 osd.2 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.886310+0000 front 2019-12-09T16:11:50.018226+0000 (oldest deadline 2019-12-09T16:12:33.573405+0000) 2019-12-09T16:13:23.282 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:13:23.275+0000 1694f700 -1 osd.7 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.344432+0000 front 2019-12-09T16:11:48.337722+0000 (oldest deadline 2019-12-09T16:12:31.857386+0000) 2019-12-09T16:13:23.399 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:23.367+0000 1694f700 -1 osd.6 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:12:08.766224+0000 (oldest deadline 2019-12-09T16:12:48.766224+0000) 2019-12-09T16:13:23.414 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:13:23.394+0000 1694f700 -1 osd.6 305 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e304) 2019-12-09T16:13:23.429 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:13:23.424+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e305 v305) 2019-12-09T16:13:23.474 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:13:23.466+0000 1694f700 -1 osd.5 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.345163+0000 front 2019-12-09T16:11:48.137115+0000 (oldest deadline 2019-12-09T16:12:32.587186+0000) 2019-12-09T16:13:23.546 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:23.540+0000 1694f700 -1 osd.0 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:48.714911+0000 front 2019-12-09T16:11:48.511388+0000 (oldest deadline 2019-12-09T16:12:32.929626+0000) 2019-12-09T16:13:23.547 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:23.543+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:23.737 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:13:23.732+0000 1694f700 -1 osd.4 305 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:11:43.262051+0000 front 2019-12-09T16:11:43.260799+0000 (oldest deadline 2019-12-09T16:12:32.468340+0000) 2019-12-09T16:13:24.597 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:24.587+0000 1694f700 -1 osd.0 306 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:25.681 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:13:25.661+0000 1694f700 -1 osd.0 307 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:117243 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5038612784 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-09T16:13:26.586 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:13:26.490+0000 20e25700 -1 RGWWatcher::handle_error cookie 5038612784 err (107) Transport endpoint is not connected 2019-12-09T16:15:00.969 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:00.964+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:01.986 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:01.980+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:02.213 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:02.206+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:02.947 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:02.941+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:03.181 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:03.174+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:03.339 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:03.334+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:03.670 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:03.665+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:03.924 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:03.919+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:03.974 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:03.970+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:04.208 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:04.201+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:04.319 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:04.314+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:04.607 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:04.601+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:04.716 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:04.711+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:04.910 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:04.905+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:05.004 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:04.999+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:05.246 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:05.240+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:05.300 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:05.294+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:05.657 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:05.652+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:05.729 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:05.722+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:05.752 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:05.747+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:05.894 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:05.890+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:06.001 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:05.995+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:06.270 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:06.264+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:06.287 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:06.281+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:06.692 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:06.687+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:06.725 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:06.719+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:06.761 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:06.755+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:06.919 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:06.914+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:07.045 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:07.040+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:07.243 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:07.238+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:07.247 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:07.242+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:07.731 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:07.726+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:07.745 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:07.740+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:07.765 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:07.760+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:07.948 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:07.944+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:08.016 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:08.010+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:08.224 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:08.218+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:08.262 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:08.255+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:08.706 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:08.701+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:08.755 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:08.750+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:08.783 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:08.778+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:08.962 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:08.956+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:09.064 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:09.059+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:09.220 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:09.212+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:09.251 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:09.246+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:09.712 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:09.707+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:09.778 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:09.773+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:09.803 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:09.797+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:09.987 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:09.982+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:10.025 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:10.020+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:10.216 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:10.210+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:10.309 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:10.303+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:10.732 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:10.727+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:10.751 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:10.747+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:10.801 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:10.795+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:10.948 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:10.943+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:11.071 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:11.067+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:11.198 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:11.192+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:11.360 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:11.354+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:11.765 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:11.760+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:11.777 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:11.772+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:11.817 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:11.805+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:11.989 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:11.985+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:12.075 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:12.070+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:12.218 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:12.213+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:12.340 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:12.336+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:12.732 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:12.726+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:12.792 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:12.788+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:12.806 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:12.800+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:12.981 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:12.977+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:13.095 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:13.090+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:13.251 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:13.246+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:13.340 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:13.334+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:13.789 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:13.762+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:13.792 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:13.787+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:13.804 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:13.799+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:13.957 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:13.953+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:14.134 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:14.128+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:14.278 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:14.272+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:14.309 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:14.304+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:14.766 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:14.760+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:14.780 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:14.775+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:14.819 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:14.813+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:14.964 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:14.958+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:15.089 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:15.085+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:15.235 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:15.229+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:15.291 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:15.286+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:15.762 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:15.758+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:15.790 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:15.784+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:15.843 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:15.837+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:15.962 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:15.957+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:16.123 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:16.117+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:16.260 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:16.254+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:16.323 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:16.317+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:16.762 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:16.756+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:16.813 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:16.808+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:16.876 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:16.871+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:16.948 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:16.943+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:17.078 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:17.073+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:17.245 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:17.238+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:17.340 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:17.334+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:17.814 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:17.808+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:17.859 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:17.846+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:17.863 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:17.858+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:17.958 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:17.953+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:18.054 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:18.049+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:18.283 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:18.277+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:18.381 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:18.376+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:18.846 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:18.840+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:18.889 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:18.884+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:18.911 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:18.906+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:18.951 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:18.947+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:19.049 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:19.044+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:19.324 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:19.318+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:19.432 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:19.427+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:19.868 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:19.863+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:19.868 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:19.863+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:19.934 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:19.930+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:19.949 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:19.944+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:20.035 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:20.030+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:20.358 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:20.353+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:20.421 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:20.415+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:20.874 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:20.869+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:20.902 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:20.897+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:20.926 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:20.921+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:20.955 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:20.950+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:21.023 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:21.018+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:21.344 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:21.338+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:21.431 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:21.424+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:21.891 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:21.886+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:21.894 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:21.890+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:21.909 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:21.905+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:21.976 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:21.971+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:22.021 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:22.016+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:22.339 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:22.333+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:22.418 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:22.413+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:22.866 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:22.862+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:22.935 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:22.931+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:22.936 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:22.930+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:22.978 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:22.972+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:23.011 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:23.006+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:23.353 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:23.347+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:23.442 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:23.437+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:23.856 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:23.850+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:23.915 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:23.908+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:23.940 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:23.935+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:23.967 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:23.963+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:24.023 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:24.018+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:24.378 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:24.372+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:24.462 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:24.455+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:24.902 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:24.897+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:24.922 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:24.916+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:24.980 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:24.975+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:25.001 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:24.996+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:25.015 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:25.010+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:25.388 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:25.383+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:25.471 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:25.465+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:25.877 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:25.872+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:25.975 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:25.969+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:25.994 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:25.989+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:26.013 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:26.008+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:26.016 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:26.010+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:26.419 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:26.414+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:26.455 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:26.449+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:26.837 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:26.832+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:26.952 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:26.947+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:27.002 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:26.997+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:27.044 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:27.034+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:27.045 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:27.038+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:27.392 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:27.386+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:27.465 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:27.460+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:27.847 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:27.842+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:27.986 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:27.981+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:28.008 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:28.003+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:28.058 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:28.053+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:28.077 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:28.073+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:28.409 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:28.405+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:28.426 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:28.420+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:28.865 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:28.860+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:29.026 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:29.022+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:29.033 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:29.029+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:29.060 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:29.055+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:29.061 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:29.057+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:29.432 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:29.427+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:29.446 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:29.440+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:29.841 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:29.836+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:30.012 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:30.005+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:30.054 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:30.049+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:30.056 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:30.051+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:30.062 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:30.057+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:30.402 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:30.397+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:30.492 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:30.487+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:30.887 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:30.881+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:31.018 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:31.013+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:31.065 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:31.059+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:31.088 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:31.083+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:31.101 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:31.093+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:31.378 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:31.372+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:31.503 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:31.497+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:31.920 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:31.915+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:32.038 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:32.034+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:32.073 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:32.068+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:32.095 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:32.089+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:32.122 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:32.116+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:32.381 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:32.376+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:32.462 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:32.456+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:32.901 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:32.896+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:33.076 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:33.071+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:33.077 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:33.071+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:33.150 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:33.144+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:33.167 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:33.162+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:33.379 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:33.373+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:33.437 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:33.432+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:33.896 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:33.891+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:33.973 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:15:33.967+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 44sec e310 v310) 2019-12-09T16:15:34.066 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:34.062+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:34.104 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:34.099+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:34.135 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:34.130+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:34.168 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:34.163+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:34.430 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:34.425+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:34.489 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:34.484+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:34.913 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:34.908+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:35.090 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:35.086+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:35.111 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:35.105+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:35.143 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:35.139+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:35.182 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:35.178+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:35.461 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:35.455+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:35.500 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:35.494+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:35.941 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:35.931+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:36.117 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:36.112+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:36.133 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:36.128+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:36.150 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:36.145+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:36.205 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:36.199+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:36.469 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:36.463+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:36.543 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:36.538+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:36.900 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:36.896+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:37.123 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:37.117+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:37.125 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:37.119+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:37.141 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:37.136+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:37.179 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:37.174+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:37.487 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:37.481+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:37.502 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:37.496+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:37.942 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:37.936+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:38.131 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:38.126+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:38.149 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:38.143+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:38.151 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:38.145+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:38.200 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:38.196+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:38.462 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:38.456+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:38.495 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:38.490+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:38.918 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:38.914+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:38.992 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:15:38.988+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 44sec e310 v310) 2019-12-09T16:15:39.133 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:39.128+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:39.170 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:39.164+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:39.175 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:39.170+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:39.250 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:39.245+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:39.464 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:39.458+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:39.494 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:39.489+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:39.884 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:15:39.879+0000 1694f700 -1 osd.0 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683877+0000 front 2019-12-09T16:14:21.382781+0000 (oldest deadline 2019-12-09T16:15:02.893666+0000) 2019-12-09T16:15:40.141 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:40.136+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:40.183 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:40.179+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:40.206 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:40.200+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:40.237 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:40.232+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:40.422 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:40.417+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:40.529 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:40.524+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:41.112 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:41.105+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:41.205 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:41.198+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:41.207 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:41.202+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:41.212 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:41.205+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:41.418 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:41.414+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:41.517 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:41.512+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:42.145 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:42.124+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:42.189 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:42.183+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:42.212 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:42.208+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:42.233 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:42.228+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:42.461 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:42.458+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:42.536 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:42.532+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:43.101 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:43.095+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:43.155 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:43.149+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:43.227 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:15:43.221+0000 1694f700 -1 osd.2 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.635813+0000 front 2019-12-09T16:14:20.881783+0000 (oldest deadline 2019-12-09T16:15:03.940824+0000) 2019-12-09T16:15:43.250 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:43.245+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:15:43.492 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:15:43.488+0000 1694f700 -1 osd.7 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:20.719334+0000 front 2019-12-09T16:14:20.819765+0000 (oldest deadline 2019-12-09T16:15:02.497657+0000) 2019-12-09T16:15:43.580 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:15:43.575+0000 1694f700 -1 osd.5 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.923146+0000 front 2019-12-09T16:14:22.159226+0000 (oldest deadline 2019-12-09T16:15:01.640764+0000) 2019-12-09T16:15:44.075 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:15:44.068+0000 1694f700 -1 osd.1 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:18.683876+0000 front 2019-12-09T16:14:23.789787+0000 (oldest deadline 2019-12-09T16:15:03.839640+0000) 2019-12-09T16:15:44.077 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:15:44.073+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 44sec e310 v310) 2019-12-09T16:15:44.198 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:15:44.193+0000 1694f700 -1 osd.3 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:16.481709+0000 front 2019-12-09T16:14:21.794471+0000 (oldest deadline 2019-12-09T16:15:00.184544+0000) 2019-12-09T16:15:44.248 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:15:44.243+0000 1694f700 -1 osd.4 310 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:14:21.745972+0000 front 2019-12-09T16:14:22.521018+0000 (oldest deadline 2019-12-09T16:15:05.326848+0000) 2019-12-09T16:16:28.585 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:04:04.849 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:16:28.598 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:04:04.862 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:16:28.605 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:04:04.827 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:16:28.610 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:04:04.833 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:16:28.619 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:04:04.840 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:16:28.625 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:04:04.847 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:16:28.864 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:04:05.128 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:16:28.884 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:04:05.106 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:16:28.913 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:04:05.135 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:16:36.897 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:36.856+0000 3927e700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T16:16:37.086 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:37.082+0000 3a280700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T16:16:37.330 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:37.274+0000 3aa81700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T16:16:37.343 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:37.313+0000 3727a700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T16:16:37.402 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:37.396+0000 3aa81700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T16:16:44.671 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:44.665+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:45.704 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:45.697+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:45.774 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:45.767+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:45.937 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:45.930+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:45.962 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:45.955+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:46.404 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:46.398+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:46.672 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:46.668+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:46.707 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:46.701+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:46.763 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:46.756+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:46.964 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:46.957+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:47.002 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:46.995+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:47.420 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:47.415+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:47.421 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:47.417+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:47.659 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:47.653+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:47.712 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:47.704+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:47.769 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:47.762+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:47.963 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:47.958+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:48.034 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:48.027+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:48.467 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:48.461+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:48.468 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:48.464+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:48.636 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:48.630+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:48.747 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:48.739+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:48.768 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:48.760+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:48.998 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:48.992+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:49.007 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:49.000+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:49.504 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:49.497+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:49.505 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:49.501+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:49.684 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:49.677+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:49.731 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:49.723+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:49.737 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:49.732+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:50.000 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:49.993+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:50.042 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:50.036+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:50.510 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:50.504+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:50.511 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:50.507+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:50.697 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:50.691+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:50.709 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:50.705+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:50.725 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:50.718+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:50.978 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:50.970+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:51.064 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:51.058+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:51.498 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:51.492+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:51.499 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:51.494+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:51.656 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:51.649+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:51.682 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:51.676+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:51.733 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:51.726+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:51.940 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:51.933+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:52.021 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:52.014+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:52.471 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:52.465+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:52.478 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:52.474+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:52.671 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:52.666+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:52.705 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:52.701+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:52.782 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:52.775+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:52.915 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:52.908+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:53.044 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:53.037+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:53.435 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:53.430+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:53.441 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:53.435+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:53.652 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:53.645+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:53.755 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:53.750+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:53.817 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:53.811+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:53.905 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:53.898+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:54.040 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:54.031+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:54.474 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:54.470+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:54.476 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:54.473+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:54.651 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:54.647+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:54.789 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:54.780+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:54.847 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:54.841+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:54.881 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:54.874+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:55.039 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:55.032+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:55.433 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:55.427+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:55.433 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:55.429+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:55.702 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:55.697+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:55.779 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:55.773+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:55.850 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:55.845+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:55.898 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:55.892+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:56.020 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:56.014+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:56.479 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:56.473+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:56.480 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:56.476+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:56.680 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:56.675+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:56.742 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:56.737+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:56.875 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:56.870+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:56.947 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:56.941+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:57.061 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:57.055+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:57.435 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:57.430+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:57.438 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:57.433+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:57.647 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:57.642+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:57.740 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:57.735+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:57.878 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:57.872+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:57.972 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:57.968+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:58.040 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:58.033+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:58.410 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:58.403+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:58.410 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:58.406+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:58.645 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:58.641+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:58.722 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:58.717+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:58.905 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:58.898+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:58.999 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:58.993+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:16:59.089 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:16:59.083+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:16:59.394 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:59.386+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:16:59.395 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:16:59.390+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:16:59.666 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:16:59.659+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:16:59.729 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:16:59.725+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:16:59.868 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:16:59.862+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:16:59.981 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:16:59.975+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:00.045 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:00.039+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:00.393 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:00.387+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:00.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:00.392+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:00.647 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:00.643+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:00.743 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:00.737+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:00.902 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:00.897+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:00.942 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:00.936+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:01.078 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:01.073+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:01.446 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:01.440+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:01.450 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:01.446+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:01.702 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:01.697+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:01.706 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:01.702+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:01.878 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:01.873+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:01.941 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:01.935+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:02.064 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:02.058+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:02.476 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:02.468+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:02.482 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:02.473+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:02.675 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:02.670+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:02.676 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:02.670+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:02.896 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:02.890+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:02.902 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:02.895+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:03.055 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:03.049+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:03.472 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:03.465+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:03.472 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:03.468+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:03.639 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:03.633+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:03.669 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:03.662+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:03.872 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:03.864+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:03.949 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:03.943+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:04.099 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:04.093+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:04.489 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:04.484+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:04.490 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:04.485+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:04.642 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:04.636+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:04.663 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:04.656+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:04.918 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:04.910+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:04.919 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:04.912+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:05.108 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:05.102+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:05.471 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:05.464+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:05.472 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:05.467+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:05.643 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:05.638+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:05.678 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:05.673+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:05.915 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:05.906+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:05.915 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:05.907+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:06.092 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:06.086+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:06.495 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:06.489+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:06.499 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:06.495+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:06.660 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:06.656+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:06.675 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:06.670+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:06.895 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:06.889+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:06.955 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:06.949+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:07.066 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:07.060+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:07.501 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:07.493+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:07.502 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:07.497+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:07.634 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:07.628+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:07.703 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:07.697+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:07.884 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:07.878+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:07.984 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:07.978+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:08.072 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:08.066+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:08.364 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:17:08.358+0000 1694f700 -1 osd.6 316 get_health_metrics reporting 48 slow ops, oldest is osd_op(client.4355.0:111706 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e314) 2019-12-09T16:17:08.515 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:08.510+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:08.516 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:08.511+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:08.648 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:08.642+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:08.665 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:08.659+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:08.842 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:08.836+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:09.018 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:09.012+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:09.073 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:09.067+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:09.447 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:17:09.431+0000 1694f700 -1 osd.6 316 get_health_metrics reporting 149 slow ops, oldest is osd_op(client.4355.0:111706 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e314) 2019-12-09T16:17:09.472 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:09.467+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:09.474 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:09.470+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:09.701 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:09.696+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:09.711 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:09.706+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:09.887 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:09.883+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:09.980 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:09.972+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:10.127 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:10.121+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:10.459 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:17:10.440+0000 1694f700 -1 osd.6 316 get_health_metrics reporting 158 slow ops, oldest is osd_op(client.4355.0:111706 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e314) 2019-12-09T16:17:10.475 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:10.467+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:10.476 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:10.472+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:10.696 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:10.691+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:10.723 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:10.718+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:10.935 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:10.929+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:10.941 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:10.935+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:11.096 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:11.091+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:11.431 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:11.426+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:11.435 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:11.431+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:11.518 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:17:11.501+0000 1694f700 -1 osd.6 316 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4355.0:111706 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e314) 2019-12-09T16:17:11.706 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:11.699+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:11.743 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:11.738+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:11.943 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:11.936+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:11.978 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:11.967+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:12.142 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:12.135+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:12.431 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:12.425+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:12.432 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:12.428+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:12.494 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:17:12.488+0000 1694f700 -1 osd.6 316 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4355.0:111706 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e314) 2019-12-09T16:17:12.667 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:12.662+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:12.762 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:12.757+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:12.926 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:12.920+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:12.947 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:12.941+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:13.188 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:13.180+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:13.420 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:13.413+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:13.425 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:13.420+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:13.522 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:17:13.506+0000 1694f700 -1 osd.6 316 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4355.0:111706 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e314) 2019-12-09T16:17:13.652 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:13.647+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:13.813 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:13.808+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:13.952 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:17:13.945+0000 1694f700 -1 osd.7 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.411170+0000 front 2019-12-09T16:16:00.394236+0000 (oldest deadline 2019-12-09T16:16:44.980849+0000) 2019-12-09T16:17:13.985 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:17:13.978+0000 1694f700 -1 osd.5 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:15:59.399652+0000 front 2019-12-09T16:15:59.409962+0000 (oldest deadline 2019-12-09T16:16:45.274585+0000) 2019-12-09T16:17:14.202 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:17:14.195+0000 1694f700 -1 osd.4 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.101805+0000 front 2019-12-09T16:16:00.092302+0000 (oldest deadline 2019-12-09T16:16:45.053913+0000) 2019-12-09T16:17:14.418 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:14.413+0000 1694f700 -1 osd.0 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.154188+0000 front 2019-12-09T16:16:00.102622+0000 (oldest deadline 2019-12-09T16:16:45.856679+0000) 2019-12-09T16:17:14.421 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:14.417+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:14.558 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:17:14.536+0000 1694f700 -1 osd.6 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:16:34.482811+0000 (oldest deadline 2019-12-09T16:17:14.482811+0000) 2019-12-09T16:17:14.572 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:17:14.553+0000 1694f700 -1 osd.6 316 get_health_metrics reporting 162 slow ops, oldest is osd_op(client.4355.0:111706 7.d 7.2d4be67d (undecoded) ondisk+retry+write+known_if_redirected e314) 2019-12-09T16:17:14.638 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:17:14.633+0000 1694f700 -1 osd.2 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:00.874561+0000 front 2019-12-09T16:16:00.668284+0000 (oldest deadline 2019-12-09T16:16:45.940153+0000) 2019-12-09T16:17:14.823 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:17:14.817+0000 1694f700 -1 osd.3 316 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:16:01.424509+0000 front 2019-12-09T16:16:01.377436+0000 (oldest deadline 2019-12-09T16:16:43.785154+0000) 2019-12-09T16:17:15.803 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:15.685+0000 1694f700 -1 osd.0 317 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:16.775 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:17:16.755+0000 1694f700 -1 osd.0 318 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:117638 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5444300992 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T16:17:17.254 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:17:17.242+0000 20e25700 -1 RGWWatcher::handle_error cookie 5444300992 err (107) Transport endpoint is not connected 2019-12-09T16:19:06.055 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:06.049+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:07.011 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:07.006+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:07.332 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:07.326+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:07.933 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:07.928+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:07.994 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:07.989+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:08.338 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:08.333+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:08.959 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:08.953+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:08.988 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:08.982+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:09.345 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:09.339+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:09.703 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:09.697+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:09.846 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:09.839+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:09.919 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:09.915+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:10.043 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:10.035+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:10.176 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:10.171+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:10.351 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:10.346+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:10.639 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:06:46.903 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:10.652 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:06:46.916 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:10.695 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:06:46.917 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:10.702 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:06:46.924 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:10.717 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:10.713+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:10.746 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:06:46.968 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:10.753 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:06:46.972 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:10.873 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:10.866+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:10.894 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:06:47.158 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:10.923 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:10.919+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:10.960 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:06:47.182 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:10.970 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:06:47.192 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:11.014 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:11.009+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:11.227 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:11.222+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:11.238 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:11.232+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:11.333 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:11.329+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:11.702 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:11.697+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:11.913 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:11.909+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:11.917 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:11.911+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:12.047 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:12.041+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:12.242 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:12.235+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:12.280 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:12.276+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:12.300 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:12.295+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:12.725 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:12.721+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:12.879 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:12.873+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:12.888 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:12.883+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:13.075 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:13.069+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:13.227 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:13.222+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:13.307 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:13.301+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:13.339 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:13.334+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:13.688 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:13.683+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:13.893 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:13.886+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:13.941 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:13.936+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:14.097 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:14.090+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:14.207 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:14.202+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:14.309 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:14.305+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:14.327 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:14.322+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:14.724 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:14.717+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:14.937 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:14.931+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:14.957 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:14.952+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:15.116 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:15.110+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:15.185 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:15.178+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:15.336 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:15.331+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:15.379 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:15.375+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:15.695 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:15.690+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:15.958 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:15.951+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:16.003 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:15.998+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:16.116 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:16.110+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:16.165 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:16.160+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:16.360 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:16.355+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:16.390 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:16.386+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:16.659 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:16.655+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:16.929 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:16.923+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:16.978 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:16.973+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:17.083 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:17.073+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:17.169 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:17.164+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:17.403 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:17.396+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:17.404 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:17.399+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:17.706 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:17.701+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:17.888 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:17.883+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:17.935 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:17.930+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:18.040 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:18.034+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:18.180 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:18.172+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:18.392 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:18.385+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:18.402 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:18.394+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:18.757 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:18.752+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:18.865 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:18.859+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:18.955 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:18.949+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:19.016 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:19.010+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:19.185 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:19.179+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:19.398 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:19.392+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:19.411 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:19.405+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:19.720 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:19.715+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:19.897 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:19.891+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:19.947 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:19.941+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:20.004 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:19.993+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:20.195 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:20.188+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:20.396 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:20.390+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:20.401 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:20.397+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:20.703 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:20.698+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:20.879 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:20.872+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:20.919 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:20.914+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:20.954 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:20.949+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:21.175 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:21.169+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:21.371 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:21.366+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:21.399 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:21.394+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:21.755 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:21.749+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:21.842 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:21.836+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:21.900 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:21.895+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:21.978 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:21.972+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:22.191 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:22.185+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:22.389 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:22.375+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:22.450 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:22.445+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:22.794 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:22.787+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:22.884 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:22.878+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:22.917 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:22.912+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:23.007 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:23.001+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:23.188 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:23.183+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:23.387 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:23.382+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:23.460 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:23.455+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:23.758 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:23.753+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:23.842 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:23.836+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:23.892 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:23.887+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:23.972 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:23.966+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:24.180 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:24.174+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:24.417 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:24.412+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:24.441 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:24.433+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:24.808 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:24.802+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:24.825 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:24.820+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:24.856 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:24.851+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:25.005 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:24.999+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:25.205 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:25.199+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:25.447 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:25.442+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:25.490 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:25.486+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:25.795 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:25.790+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:25.841 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:25.836+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:25.842 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:25.837+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:26.010 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:26.005+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:26.205 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:26.200+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:26.444 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:26.440+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:26.511 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:26.506+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:26.780 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:26.776+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:26.878 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:26.873+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:26.879 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:26.873+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:27.003 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:26.997+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:27.189 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:27.185+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:27.498 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:27.492+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:27.558 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:27.553+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:27.741 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:27.735+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:27.890 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:27.886+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:27.934 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:27.929+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:28.008 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:28.003+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:28.182 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:28.178+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:28.496 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:28.492+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:28.534 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:28.528+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:28.702 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:28.697+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:28.933 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:28.926+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:28.986 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:28.981+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:29.025 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:29.020+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:29.165 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:29.161+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:29.516 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:29.509+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:29.517 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:29.512+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:29.685 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:29.680+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:29.906 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:29.901+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:29.941 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:29.935+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:30.067 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:30.062+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:30.205 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:30.200+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:30.518 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:30.513+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:30.565 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:30.560+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:30.662 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:30.658+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:30.863 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:30.858+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:30.941 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:30.935+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:31.067 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:31.062+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:31.233 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:31.229+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:31.552 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:31.546+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:31.600 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:31.595+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:31.686 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:31.681+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:31.893 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:31.888+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:31.947 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:31.941+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:32.051 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:32.046+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:32.218 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:32.213+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:32.565 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:32.558+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:32.614 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:32.608+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:32.681 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:32.676+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:32.881 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:32.876+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:32.900 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:32.895+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:33.064 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:33.058+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:33.222 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:33.216+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:33.555 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:33.550+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:33.579 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:33.574+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:33.701 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:33.696+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:33.879 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:33.874+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:33.951 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:33.945+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:34.103 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:34.098+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:34.216 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:34.211+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:34.543 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:34.537+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:34.591 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:34.586+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:34.683 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:34.679+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:34.856 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:34.851+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:34.981 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:34.976+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:35.151 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:35.146+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:35.258 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:35.254+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:35.561 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:35.556+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:35.627 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:35.623+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:35.715 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:35.711+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:35.894 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:35.889+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:35.990 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:35.985+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:36.117 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:36.112+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:36.301 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:36.295+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:36.576 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:36.571+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:36.653 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:36.648+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:36.748 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:36.743+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:36.884 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:36.879+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:37.007 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:37.002+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:37.102 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:37.097+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:37.305 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:37.301+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:37.612 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:37.607+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:37.706 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:37.701+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:37.722 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:37.717+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:37.927 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:37.921+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:38.000 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:37.995+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:38.116 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:38.111+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:38.342 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:38.337+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:38.643 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:38.634+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:38.719 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:38.714+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:38.741 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:38.721+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:38.965 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:38.954+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:38.966 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:38.961+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:39.076 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:39.071+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:39.330 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:39.326+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:39.680 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:39.675+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:39.694 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:39.689+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:39.721 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:39.717+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:39.949 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:39.943+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:39.996 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:39.991+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:40.064 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:40.059+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:40.375 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:40.371+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:40.568 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:19:40.564+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 41sec e321 v321) 2019-12-09T16:19:40.663 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:40.658+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:40.686 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:40.681+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:40.760 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:40.755+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:40.908 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:40.904+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:41.033 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:41.022+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:41.071 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:41.066+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:41.416 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:41.412+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:41.668 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:41.664+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:41.700 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:41.695+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:41.781 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:41.776+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:41.930 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:41.925+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:42.038 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:42.033+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:42.048 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:42.044+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:42.376 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:42.370+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:42.660 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:42.655+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:42.726 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:42.720+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:42.801 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:42.796+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:42.975 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:42.971+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:43.026 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:43.020+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:43.088 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:43.082+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:43.425 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:43.420+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:43.679 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:43.674+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:43.739 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:43.732+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:43.846 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:43.840+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:43.985 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:43.978+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:44.017 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:44.010+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:44.052 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:44.046+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:44.478 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:44.472+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:44.718 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:44.712+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:44.765 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:44.756+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:44.866 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:44.860+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:44.950 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:44.943+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:45.027 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:45.021+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:45.092 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:45.087+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:45.475 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:45.469+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:45.582 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:19:45.577+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 41sec e321 v321) 2019-12-09T16:19:45.699 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:45.693+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:45.729 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:45.723+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:45.915 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:45.908+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:45.924 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:45.918+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:45.999 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:45.993+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:46.066 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:46.060+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:46.454 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:46.448+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:46.711 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:46.703+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:46.774 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:46.769+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:46.889 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:46.883+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:46.957 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:46.952+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:47.020 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:47.016+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:47.045 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:47.039+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:47.476 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:47.470+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:47.743 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:47.738+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:47.752 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:47.747+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:47.913 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:47.907+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:47.979 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:47.974+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:48.073 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:48.068+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:48.076 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:48.070+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:48.506 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:48.501+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:48.734 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:48.730+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:48.754 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:48.750+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:48.940 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:48.934+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:49.029 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:49.025+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:49.112 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:49.106+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:49.122 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:49.117+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:49.559 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:49.554+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:49.720 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:49.714+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:49.764 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:49.760+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:49.914 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:49.909+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:50.015 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:50.010+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:50.086 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:50.080+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:50.123 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:50.118+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:50.594 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:50.583+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:50.650 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:19:50.645+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 41sec e321 v321) 2019-12-09T16:19:50.749 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:50.744+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:50.753 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:50.749+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:50.924 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:50.919+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:51.059 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:51.053+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:51.119 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:51.114+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:51.154 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:51.148+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:51.552 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:51.547+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:51.777 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:51.771+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:51.799 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:51.795+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:51.893 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:51.886+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:52.035 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:52.026+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:52.137 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:52.127+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:52.166 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:52.160+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:52.550 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:52.544+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:52.740 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:52.731+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:52.834 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:52.830+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:52.883 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:52.877+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:53.083 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:53.072+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:53.109 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:53.103+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:53.149 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:53.143+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:53.515 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:53.509+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:53.728 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:53.723+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:53.796 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:53.790+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:53.914 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:53.908+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:54.040 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:54.035+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:54.147 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:54.141+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:54.185 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:54.180+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:54.521 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:54.516+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:54.772 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:54.767+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:54.823 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:54.818+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:54.915 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:54.909+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:55.094 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:19:55.089+0000 1694f700 -1 osd.1 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.272393+0000 front 2019-12-09T16:18:25.271418+0000 (oldest deadline 2019-12-09T16:19:09.315499+0000) 2019-12-09T16:19:55.132 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:19:55.127+0000 1694f700 -1 osd.4 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:29.730227+0000 front 2019-12-09T16:18:24.704604+0000 (oldest deadline 2019-12-09T16:19:05.274023+0000) 2019-12-09T16:19:55.178 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:55.173+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:19:55.558 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:19:55.552+0000 1694f700 -1 osd.7 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:25.322137+0000 front 2019-12-09T16:18:25.316413+0000 (oldest deadline 2019-12-09T16:19:10.557292+0000) 2019-12-09T16:19:55.739 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:19:55.728+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 41sec e321 v321) 2019-12-09T16:19:55.778 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:19:55.774+0000 1694f700 -1 osd.3 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:30.818821+0000 front 2019-12-09T16:18:27.049102+0000 (oldest deadline 2019-12-09T16:19:06.940382+0000) 2019-12-09T16:19:55.801 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:19:55.792+0000 1694f700 -1 osd.0 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:24.705223+0000 front 2019-12-09T16:18:24.417447+0000 (oldest deadline 2019-12-09T16:19:09.175464+0000) 2019-12-09T16:19:55.952 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:19:55.947+0000 1694f700 -1 osd.5 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:23.582406+0000 front 2019-12-09T16:18:30.938809+0000 (oldest deadline 2019-12-09T16:19:09.408097+0000) 2019-12-09T16:19:56.160 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:19:56.155+0000 1694f700 -1 osd.2 321 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:18:27.445438+0000 front 2019-12-09T16:18:26.677176+0000 (oldest deadline 2019-12-09T16:19:07.240253+0000) 2019-12-09T16:20:20.988 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:07:57.252 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:21.040 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:07:57.305 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:21.073 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:07:57.294 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:21.075 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:07:57.296 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:21.075 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:07:57.296 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:21.085 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:07:57.303 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:21.137 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:07:57.401 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:21.182 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:07:57.404 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:21.194 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:07:57.416 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:23.058 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:07:59.323 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:23.069 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:07:59.332 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:23.083 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:07:59.305 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:23.084 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:07:59.307 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:23.094 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:07:59.315 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:23.095 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:07:59.317 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:23.110 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:07:59.375 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:23.125 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:07:59.346 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:23.137 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:07:59.359 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:20:50.903 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:50.898+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:20:51.926 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:51.921+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:20:52.071 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:20:52.066+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:20:52.723 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:20:52.718+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:20:52.978 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:52.972+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:20:52.983 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:52.978+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:20:53.117 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:20:53.111+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:20:53.752 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:20:53.748+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:20:53.998 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:53.992+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:20:53.999 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:53.995+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:20:54.119 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:20:54.113+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:20:54.442 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:20:54.435+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:20:54.739 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:20:54.733+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:20:54.963 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:20:54.958+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:20:54.998 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:54.991+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:20:54.998 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:54.994+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:20:55.161 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:20:55.153+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:20:55.463 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:20:55.457+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:20:55.701 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:20:55.696+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:20:55.797 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:20:55.791+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:20:55.980 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:55.974+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:20:55.980 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:55.977+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:20:56.018 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:20:56.011+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:20:56.142 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:20:56.136+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:20:56.442 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:20:56.436+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:20:56.672 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:20:56.665+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:20:56.831 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:20:56.824+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:20:56.943 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:56.933+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:20:56.945 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:56.938+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:20:57.012 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:20:57.007+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:20:57.124 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:20:57.118+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:20:57.483 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:20:57.477+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:20:57.656 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:20:57.650+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:20:57.825 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:20:57.817+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:20:57.985 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:57.979+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:20:57.985 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:57.981+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:20:58.061 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:20:58.055+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:20:58.166 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:20:58.158+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:20:58.465 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:20:58.453+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:20:58.643 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:20:58.637+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:20:58.782 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:20:58.775+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:20:58.963 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:58.956+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:20:58.964 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:58.960+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:20:59.056 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:20:59.050+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:20:59.154 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:20:59.146+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:20:59.476 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:20:59.469+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:20:59.666 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:20:59.660+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:20:59.824 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:20:59.816+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:20:59.955 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:59.948+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:20:59.956 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:20:59.951+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:00.062 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:00.056+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:00.135 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:21:00.126+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:21:00.496 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:00.488+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:00.702 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:00.697+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:00.850 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:00.842+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:00.932 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:00.925+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:00.933 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:00.928+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:01.056 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:01.049+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:01.159 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:21:01.152+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:21:01.461 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:01.452+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:01.731 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:01.725+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:01.828 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:01.821+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:01.944 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:01.938+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:01.949 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:01.941+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:02.090 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:02.084+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:02.161 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:21:02.154+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:21:02.520 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:02.507+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:02.759 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:02.752+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:02.814 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:02.803+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:02.821 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:02.815+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:02.961 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:02.954+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:02.961 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:02.957+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:03.127 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:03.120+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:03.197 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:21:03.187+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:21:03.499 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:03.491+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:03.751 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:03.745+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:03.856 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:03.849+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:03.895 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:03.877+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 77 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:04.008 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:04.002+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:04.010 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:04.004+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:04.120 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:04.114+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:04.203 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:21:04.197+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:20:51.449318+0000) 2019-12-09T16:21:04.477 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:04.466+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:04.764 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:04.755+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:04.866 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:04.860+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:04.916 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:04.902+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:05.021 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:05.009+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:05.021 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:05.013+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:05.080 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:05.074+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:05.482 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:05.476+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:05.793 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:05.780+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:05.835 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:05.828+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:05.932 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:05.915+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:06.033 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:06.028+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:06.036 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:06.032+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:06.081 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:06.076+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:06.510 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:06.501+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:06.802 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:06.794+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:06.869 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:06.860+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:06.978 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:06.963+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:07.034 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:07.027+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:07.035 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:07.030+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:07.072 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:07.067+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:07.476 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:07.469+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:07.819 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:07.814+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:07.921 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:07.914+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:07.955 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:07.937+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:08.055 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:08.048+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:08.065 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:08.058+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:08.065 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:08.061+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:08.465 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:08.458+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:08.856 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:08.849+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:08.894 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:08.887+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:08.904 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:08.897+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:08.934 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:08.918+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:09.034 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:09.028+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:09.109 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:09.102+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:09.110 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:09.105+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:09.455 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:09.448+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:09.841 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:09.835+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:09.869 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:09.862+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:09.929 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:09.904+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:09.946 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:09.925+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:10.020 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:10.016+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:10.133 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:10.127+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:10.135 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:10.131+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:10.492 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:10.485+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:10.818 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:10.813+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:10.891 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:10.885+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:10.942 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:10.936+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:10.960 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:10.949+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:11.003 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:10.997+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:11.188 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:11.182+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:11.189 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:11.185+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:11.478 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:11.471+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:11.832 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:11.827+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:11.918 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:11.911+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:11.933 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:11.914+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:11.945 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:11.929+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:12.023 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:12.017+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:12.181 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:12.173+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:12.182 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:12.177+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:12.458 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:12.450+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:12.808 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:12.801+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:12.885 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:12.879+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:12.981 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:12.961+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:12.991 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:12.976+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:13.064 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:13.057+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:13.195 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:13.190+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:13.198 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:13.194+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:13.457 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:13.450+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:13.813 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:13.807+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:13.861 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:13.854+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:13.947 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:13.940+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:13.978 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:13.961+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:14.105 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:14.098+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:14.155 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:14.149+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:14.159 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:14.155+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:14.458 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:14.450+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:14.767 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:14.763+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:14.820 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:14.814+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:14.961 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:14.937+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:14.975 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:14.956+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:15.144 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:15.138+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:15.200 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:15.193+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:15.202 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:15.197+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:15.466 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:15.460+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:15.761 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:15.753+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:15.827 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:15.817+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:15.921 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:15.914+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:15.951 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:15.934+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:16.160 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:16.153+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:16.161 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:16.156+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:16.196 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:16.190+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:16.485 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:16.479+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:16.714 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:16.710+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:16.835 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:16.828+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:16.975 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:16.969+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:17.009 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:16.990+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:17.169 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:17.161+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:17.170 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:17.165+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:17.250 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:17.244+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:17.498 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:17.491+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:17.688 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:17.681+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:17.874 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:17.866+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:18.043 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:18.037+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:18.074 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:18.057+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:18.217 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:18.210+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:18.219 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:18.214+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:18.266 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:18.261+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:18.503 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:18.497+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:18.654 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:18.649+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:18.882 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:18.876+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:19.106 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:19.099+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:19.136 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:19.118+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:19.233 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:19.227+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:19.272 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:19.265+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:19.274 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:19.269+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:19.547 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:19.542+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:19.706 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:19.701+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:19.845 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:19.838+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:20.173 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:20.155+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:20.184 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:20.168+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:20.256 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:20.248+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:20.256 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:20.250+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:20.257 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:20.251+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:20.557 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:20.552+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:20.738 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:20.732+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:20.839 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:20.832+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:21.158 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:21.152+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:21.177 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:21.165+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:21.239 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:21.234+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:21.303 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:21.298+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:21.305 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:21.301+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:21.572 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:21.566+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:21.758 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:21.751+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:21.869 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:21.861+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:22.169 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:22.163+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:22.195 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:22.180+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:22.235 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:22.228+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:22.329 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:22.323+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:22.332 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:22.327+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:22.531 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:22.525+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:22.769 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:22.762+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:22.852 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:22.845+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:23.178 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:23.155+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:23.185 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:23.174+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:23.228 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:23.221+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:23.382 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:23.375+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:23.384 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:23.377+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:23.576 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:23.569+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:23.741 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:23.737+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:23.900 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:23.892+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:24.176 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:24.169+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:24.210 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:24.191+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:24.265 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:24.259+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:24.340 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:24.333+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:24.341 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:24.336+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:24.542 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:24.535+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:24.787 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:24.782+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:24.902 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:24.894+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:25.254 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:25.226+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:25.264 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:25.245+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:25.280 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:25.274+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:25.382 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:25.376+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:25.384 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:25.379+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:25.572 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:25.565+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:25.818 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:25.812+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:25.868 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:25.859+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:26.190 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:21:26.180+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 42sec e327 v327) 2019-12-09T16:21:26.240 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:26.231+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:26.301 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:26.286+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:26.312 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:26.296+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:26.337 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:26.331+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:26.337 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:26.333+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:26.539 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:26.532+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:26.860 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:26.855+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:26.898 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:26.890+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:27.244 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:27.239+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:27.272 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:27.249+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:27.286 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:27.267+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:27.390 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:27.384+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:27.391 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:27.386+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:27.542 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:27.535+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:27.834 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:27.829+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:27.915 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:27.908+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:28.235 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:28.226+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:28.286 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:28.262+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:28.298 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:28.281+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:28.381 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:28.374+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:28.381 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:28.377+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:28.540 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:28.534+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:28.837 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:28.832+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:28.881 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:28.874+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:29.231 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:29.224+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:29.241 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:29.236+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:29.261 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:29.248+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:29.342 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:29.336+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:29.343 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:29.338+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:29.509 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:29.502+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:29.842 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:29.835+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:29.880 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:29.873+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:30.239 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:30.233+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:30.239 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:30.233+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:30.270 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:30.253+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:30.301 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:30.295+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:30.305 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:30.300+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:30.480 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:30.473+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:30.813 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:30.808+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:30.843 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:30.837+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:31.206 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:31.199+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:31.218 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:21:31.206+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 42sec e327 v327) 2019-12-09T16:21:31.269 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:31.253+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:31.277 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:31.271+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:31.281 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:31.277+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:31.285 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:31.265+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:31.476 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:31.468+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:31.803 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:31.797+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:31.888 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:31.881+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:32.199 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:32.193+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:32.258 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:32.252+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:32.289 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:32.271+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:32.316 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:32.311+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:32.319 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:32.315+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:32.446 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:32.438+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:32.798 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:32.789+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:32.921 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:32.914+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:33.240 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:33.233+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:33.300 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:33.275+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:33.317 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:33.295+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:33.350 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:33.343+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:33.351 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:33.347+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:33.458 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:33.451+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:33.767 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:33.761+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:33.937 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:33.930+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:34.239 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:34.233+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:34.308 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:34.300+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:34.310 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:34.304+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:34.353 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:34.346+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:34.381 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:34.365+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:34.488 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:34.482+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:34.821 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:34.815+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:34.950 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:34.943+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:35.262 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:35.255+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:35.275 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:35.266+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:35.275 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:35.267+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:35.325 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:35.318+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:35.357 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:35.339+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:35.507 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:35.501+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:35.868 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:35.863+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:35.959 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:35.952+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:36.235 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:21:36.225+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 42sec e327 v327) 2019-12-09T16:21:36.252 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:36.245+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:36.253 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:36.248+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:36.315 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:36.310+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:36.354 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:36.341+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:36.365 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:36.349+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:36.498 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:36.493+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:36.919 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:36.913+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:36.925 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:36.918+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:37.249 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:37.243+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:37.250 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:37.245+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:37.329 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:37.309+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:37.340 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:37.335+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:37.342 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:37.323+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:37.513 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:37.505+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:37.887 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:37.880+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:37.901 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:37.893+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:38.243 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:38.237+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:38.244 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:38.240+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:38.308 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:38.286+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:38.321 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:38.298+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:38.332 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:38.327+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:38.491 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:38.484+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:38.873 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:38.867+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:38.941 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:38.933+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:39.281 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:39.272+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:39.283 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:39.276+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:39.319 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:39.290+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:39.335 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:39.309+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:39.355 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:39.350+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:39.486 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:39.480+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:39.868 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:39.862+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:39.911 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:39.904+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:40.290 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:40.284+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:40.303 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:40.298+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:40.305 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:40.301+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:40.316 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:40.303+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:40.389 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:40.384+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:40.460 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:40.454+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:40.828 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:40.822+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:40.936 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:40.928+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:41.258 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:21:41.250+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 42sec e327 v327) 2019-12-09T16:21:41.265 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:41.255+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:41.268 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:41.261+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:41.336 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:41.314+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:41.345 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:41.332+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:41.352 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:41.345+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:41.465 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:41.452+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:41.802 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:41.798+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:41.950 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:41.943+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:42.306 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:42.298+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:42.307 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:42.302+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:42.319 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:42.295+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:42.332 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:42.314+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:42.376 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:42.369+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:42.445 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:42.438+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:42.817 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:42.813+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:43.008 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:43.001+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:43.281 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:43.273+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:43.282 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:43.277+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:43.335 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:43.311+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:43.335 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:43.330+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:43.348 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:43.330+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:43.418 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:43.411+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:43.794 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:43.789+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:43.982 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:43.974+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:44.314 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:44.308+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:44.325 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:44.314+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:44.341 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:44.319+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:44.354 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:44.337+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:44.381 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:44.376+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:44.429 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:44.422+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:44.794 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:44.788+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:45.016 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:45.010+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:45.338 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:45.315+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:45.351 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:45.333+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:45.357 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:45.350+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:45.364 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:45.354+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:45.389 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:45.384+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:45.439 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:45.432+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:45.842 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:45.836+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:45.988 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:45.980+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:46.281 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:21:46.272+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 42sec e327 v327) 2019-12-09T16:21:46.307 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:46.300+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:46.341 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:46.319+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:46.350 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:46.344+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:46.371 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:46.366+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:46.376 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:46.371+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:46.447 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:46.441+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:46.812 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:46.805+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:46.992 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:46.986+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:47.309 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:47.303+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:47.316 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:47.308+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:47.343 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:47.327+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:47.345 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:47.338+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:47.346 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:47.342+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:47.417 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:47.410+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:47.591 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:21:47.585+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:21:47.317163+0000) 2019-12-09T16:21:47.833 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:47.827+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:48.004 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:47.997+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:48.326 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:48.319+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:48.375 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:48.351+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:48.388 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:48.370+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:48.390 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:48.383+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:48.392 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:48.387+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:48.403 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:48.396+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:48.549 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:21:48.542+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:21:47.317163+0000) 2019-12-09T16:21:48.807 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:48.802+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:49.016 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:49.008+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:49.311 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:49.304+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:49.382 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:49.376+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:49.391 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:49.383+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:49.393 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:49.388+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:49.408 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:49.396+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:49.409 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:49.399+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:49.556 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:21:49.549+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:21:47.317163+0000) 2019-12-09T16:21:49.774 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:49.769+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:50.015 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:50.008+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:50.331 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:50.325+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:50.392 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:50.384+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:50.424 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:50.418+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:50.437 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:50.430+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:50.439 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:50.434+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:50.447 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:50.432+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:50.570 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:21:50.565+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:21:47.317163+0000) 2019-12-09T16:21:50.810 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:21:50.805+0000 1694f700 -1 osd.3 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.525455+0000 front 2019-12-09T16:20:09.509496+0000 (oldest deadline 2019-12-09T16:20:52.266672+0000) 2019-12-09T16:21:50.992 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:21:50.986+0000 1694f700 -1 osd.5 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:10.840351+0000 front 2019-12-09T16:20:10.835664+0000 (oldest deadline 2019-12-09T16:20:54.908978+0000) 2019-12-09T16:21:51.357 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:21:51.350+0000 1694f700 -1 osd.7 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.343600+0000 front 2019-12-09T16:20:11.472684+0000 (oldest deadline 2019-12-09T16:20:54.340194+0000) 2019-12-09T16:21:51.368 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:21:51.353+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 42sec e327 v327) 2019-12-09T16:21:51.374 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:21:51.369+0000 1694f700 -1 osd.2 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.967316+0000 front 2019-12-09T16:20:10.584220+0000 (oldest deadline 2019-12-09T16:20:54.762258+0000) 2019-12-09T16:21:51.403 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:51.398+0000 1694f700 -1 osd.0 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:08.335874+0000 front 2019-12-09T16:20:08.541894+0000 (oldest deadline 2019-12-09T16:20:52.502940+0000) 2019-12-09T16:21:51.407 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:51.404+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:51.488 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:51.483+0000 1694f700 -1 osd.6 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:20:28.138399+0000 (oldest deadline 2019-12-09T16:21:08.138399+0000) 2019-12-09T16:21:51.523 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:21:51.510+0000 1694f700 -1 osd.6 327 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e325) 2019-12-09T16:21:51.574 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:21:51.567+0000 1694f700 -1 osd.4 327 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:20:12.344770+0000 front 2019-12-09T16:20:05.931712+0000 (oldest deadline 2019-12-09T16:21:47.317163+0000) 2019-12-09T16:21:52.389 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:21:52.385+0000 1694f700 -1 osd.0 328 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:118025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 399705040 gen 1] snapc 0=[] ondisk+write+known_if_redirected e323) 2019-12-09T16:21:54.337 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:21:54.106+0000 20e25700 -1 RGWWatcher::handle_error cookie 399705040 err (107) Transport endpoint is not connected 2019-12-09T16:22:20.416 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:22:20.411+0000 1694f700 -1 osd.1 329 get_health_metrics reporting 1 slow ops, oldest is osd_repop(osd.0.0:29306 5.5 e327/320 5:a93a5511:::notify.2:head v 327'240) 2019-12-09T16:22:24.961 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:22:24.943+0000 3727a700 -1 osd.1 332 build_incremental_map_msg missing incremental map 332 2019-12-09T16:23:24.929 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:24.925+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:25.926 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:25.921+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:26.020 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:26.014+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:26.307 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:26.301+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:26.652 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:26.647+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:26.877 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:26.872+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:26.906 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:26.901+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:27.039 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:27.034+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:27.295 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:27.290+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:27.620 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:27.615+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:27.785 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:27.780+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:27.853 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:27.847+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:27.947 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:27.942+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:28.051 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:28.039+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:28.265 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:28.261+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:28.656 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:28.652+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:28.780 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:28.774+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:28.839 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:28.834+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:28.946 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:28.941+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:29.049 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:29.043+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:29.075 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:29.071+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:29.257 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:29.252+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:29.655 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:29.650+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:29.775 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:29.770+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:29.809 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:29.800+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:29.971 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:29.965+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:30.060 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:30.056+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:30.081 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:30.070+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:30.300 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:30.294+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:30.647 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:30.634+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:30.763 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:30.759+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:30.765 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:30.759+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:31.017 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:31.011+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:31.094 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:31.089+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:31.102 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:31.096+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:31.292 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:31.285+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:31.632 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:31.627+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:31.770 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:31.765+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:31.817 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:31.811+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:32.069 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:32.063+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:32.090 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:32.084+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:32.130 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:32.124+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:32.289 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:32.269+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:32.604 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:32.599+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:32.781 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:32.775+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:32.868 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:32.862+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:33.048 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:33.043+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:33.095 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:33.090+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:33.101 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:33.095+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:33.266 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:33.260+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:33.572 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:33.567+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:33.760 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:33.754+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:33.869 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:33.863+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:34.075 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:34.070+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:34.125 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:34.119+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:34.139 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:34.135+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:34.269 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:34.263+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:34.618 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:34.612+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:34.810 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:34.805+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:34.869 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:34.864+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:35.101 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:35.094+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:35.101 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:35.095+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:35.161 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:35.156+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:35.253 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:35.247+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:35.652 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:35.647+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:35.826 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:35.821+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:35.866 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:35.859+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:36.061 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:36.055+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:36.096 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:36.090+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:36.209 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:36.198+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:36.256 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:36.247+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:36.675 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:36.670+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:36.812 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:36.807+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:36.871 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:36.865+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:37.020 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:37.015+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:37.051 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:37.046+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:37.210 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:37.204+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:37.230 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:37.225+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:37.667 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:37.662+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:37.781 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:37.776+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:37.898 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:37.891+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:38.001 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:37.995+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:38.053 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:38.048+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:38.187 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:38.182+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:38.207 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:38.202+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:38.706 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:38.701+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:38.819 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:38.814+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:38.915 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:38.909+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:38.990 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:38.984+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:39.069 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:39.060+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:39.153 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:39.148+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:39.262 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:39.256+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:39.719 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:39.714+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:39.847 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:39.842+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:39.912 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:39.907+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:39.970 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:39.963+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:40.081 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:40.077+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:40.180 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:40.175+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:40.262 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:40.258+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:40.741 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:40.736+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:40.897 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:40.892+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:40.950 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:40.930+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:40.950 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:40.943+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:41.065 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:41.056+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:41.163 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:41.158+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:41.222 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:41.217+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:41.738 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:41.733+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:41.895 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:41.889+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:41.930 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:41.923+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:41.944 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:41.938+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:42.109 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:42.102+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:42.122 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:42.117+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:42.279 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:42.272+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:42.725 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:42.714+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:42.863 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:42.857+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:42.956 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:42.950+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:42.979 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:42.974+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:43.078 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:43.073+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:43.157 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:43.153+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:43.242 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:43.235+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:43.729 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:43.725+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:43.877 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:43.871+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:43.925 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:43.918+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:44.033 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:44.025+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:44.057 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:44.052+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:44.136 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:44.131+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:44.234 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:44.228+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:44.770 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:44.765+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:44.848 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:44.842+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:44.960 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:44.953+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:45.011 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:45.007+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:45.075 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:45.066+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:45.171 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:45.166+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:45.291 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:45.284+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:45.794 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:45.788+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:45.835 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:45.828+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:45.985 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:45.979+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:46.048 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:46.042+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:46.107 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:46.101+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:46.207 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:46.200+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:46.289 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:46.283+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:46.835 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:46.830+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:46.876 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:46.871+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:47.003 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:46.997+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:47.047 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:47.041+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:47.085 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:47.079+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:47.183 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:47.176+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:47.297 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:47.289+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:47.829 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:47.823+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:47.834 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:47.829+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:47.981 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:47.975+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:48.070 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:48.064+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:48.123 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:48.117+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:48.224 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:48.218+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:48.347 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:48.341+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:48.806 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:48.800+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:48.839 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:48.833+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:48.955 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:48.950+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:49.036 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:49.030+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:49.080 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:49.075+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:49.197 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:49.192+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:49.397 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:49.391+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:49.802 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:49.796+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:49.830 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:49.824+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:49.998 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:49.992+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:50.044 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:50.039+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:50.115 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:50.110+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:50.177 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:50.172+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:50.447 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:50.441+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:50.764 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:50.759+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:50.802 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:50.796+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:51.031 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:51.025+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:51.034 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:51.030+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:51.133 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:51.128+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:51.171 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:51.165+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:51.494 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:51.488+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:51.759 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:51.754+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:51.825 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:51.818+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:52.007 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:52.002+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:52.037 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:52.030+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:52.103 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:52.098+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:52.211 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:52.207+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:52.526 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:52.520+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:52.809 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:52.804+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:52.849 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:52.843+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:52.968 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:52.964+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:52.998 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:52.993+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:53.079 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:53.073+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:53.202 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:53.198+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:53.534 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:53.527+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:53.766 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:53.760+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:53.821 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:23:53.814+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:44.256280+0000 front 2019-12-09T16:22:44.336095+0000 (oldest deadline 2019-12-09T16:23:26.863036+0000) 2019-12-09T16:23:53.994 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:53.989+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:54.035 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:54.029+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:54.045 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:54.039+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:54.186 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:54.181+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:54.530 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:54.524+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:54.759 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:54.754+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:54.989 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:54.984+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:55.009 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:55.004+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:55.054 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:55.049+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:55.219 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:55.214+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:55.531 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:55.525+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:55.803 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:55.798+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:55.981 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:55.976+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:55.994 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:55.989+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:56.092 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:56.086+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:56.267 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:56.262+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:56.528 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:56.522+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:56.808 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:23:56.804+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 45sec e332 v332) 2019-12-09T16:23:56.843 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:56.838+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:56.979 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:56.973+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:57.029 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:57.025+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:57.100 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:57.094+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:57.250 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:57.244+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:57.490 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:57.483+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:57.874 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:57.868+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:57.982 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:57.977+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:58.033 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:58.029+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:58.088 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:58.083+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:58.279 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:58.273+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:58.481 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:58.473+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:58.826 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:58.822+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:23:59.013 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:23:59.008+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:23:59.054 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:23:59.048+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:23:59.062 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:23:59.055+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:23:59.328 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:23:59.322+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:23:59.450 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:23:59.444+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:23:59.828 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:23:59.823+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:00.027 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:00.021+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:00.094 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:00.089+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:00.106 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:00.097+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:00.352 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:00.347+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:00.406 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:00.400+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:00.796 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:00.791+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:01.064 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:01.059+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:01.099 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:01.093+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:01.123 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:01.118+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:01.361 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:01.357+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:01.423 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:01.418+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:01.807 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:01.803+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:01.829 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:24:01.821+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 45sec e332 v332) 2019-12-09T16:24:02.078 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:02.074+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:02.139 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:02.131+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:02.166 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:02.162+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:02.391 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:02.386+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:02.481 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:02.474+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:02.795 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:02.789+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:03.060 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:03.055+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:03.117 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:03.110+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:03.128 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:03.123+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:03.365 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:03.361+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:03.470 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:03.464+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:03.827 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:03.821+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:04.029 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:04.023+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:04.110 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:04.105+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:04.163 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:04.156+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:04.404 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:04.399+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:04.461 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:04.455+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:04.825 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:04.819+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:05.036 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:05.032+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:05.161 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:05.155+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:05.163 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:05.156+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:05.419 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:05.413+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:05.450 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:05.445+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:05.788 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:05.783+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:06.007 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:06.002+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:06.120 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:06.116+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:06.185 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:06.179+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:06.384 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:06.379+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:06.473 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:06.468+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:06.752 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:06.747+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:06.864 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:24:06.859+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 45sec e332 v332) 2019-12-09T16:24:07.046 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:07.040+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:07.098 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:07.093+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:07.181 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:07.175+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:07.363 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:07.357+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:07.464 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:07.458+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:07.718 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:07.714+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:08.084 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:08.080+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:08.125 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:08.120+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:08.144 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:08.139+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:08.359 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:08.353+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:08.513 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:08.508+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:08.704 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:08.699+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:09.109 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:09.104+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:09.117 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:09.113+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:09.135 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:09.129+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:09.328 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:09.322+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:09.497 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:09.492+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:09.696 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:09.691+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:10.091 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:10.085+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:10.111 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:10.106+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:10.123 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:10.118+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:10.346 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:10.341+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:10.529 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:10.524+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:10.729 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:10.724+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:11.084 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:24:11.079+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:40.457088+0000 front 2019-12-09T16:22:40.240033+0000 (oldest deadline 2019-12-09T16:23:24.626825+0000) 2019-12-09T16:24:11.125 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:11.119+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:11.129 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:11.123+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:11.335 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:24:11.329+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.542174+0000 front 2019-12-09T16:22:44.039515+0000 (oldest deadline 2019-12-09T16:23:25.980984+0000) 2019-12-09T16:24:11.572 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:24:11.567+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:45.280977+0000 front 2019-12-09T16:22:44.482598+0000 (oldest deadline 2019-12-09T16:23:28.413945+0000) 2019-12-09T16:24:11.697 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:24:11.692+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.621626+0000 front 2019-12-09T16:22:40.603426+0000 (oldest deadline 2019-12-09T16:23:26.437793+0000) 2019-12-09T16:24:11.941 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:24:11.932+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 45sec e332 v332) 2019-12-09T16:24:12.106 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:24:12.101+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:43.981929+0000 front 2019-12-09T16:22:44.049586+0000 (oldest deadline 2019-12-09T16:23:25.525972+0000) 2019-12-09T16:24:12.125 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:24:12.120+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:22:41.784694+0000 front 2019-12-09T16:22:41.655498+0000 (oldest deadline 2019-12-09T16:23:27.525490+0000) 2019-12-09T16:24:19.159 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:11:55.424 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:24:19.169 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:11:55.433 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:24:19.179 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:11:55.401 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:24:19.182 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:11:55.404 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:24:19.189 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:11:55.411 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:24:19.193 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:11:55.415 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:24:19.440 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:11:55.704 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:24:19.510 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:11:55.732 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:24:19.519 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:11:55.740 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:25:07.271 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:07.266+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:08.323 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:08.319+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:09.310 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:09.306+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:09.449 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:09.442+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:09.469 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:09.454+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:09.770 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:09.764+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:10.364 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:10.360+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:10.418 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:10.412+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:10.496 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:10.488+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:10.750 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:10.743+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:11.366 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:11.363+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:11.406 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:11.399+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:11.542 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:11.534+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:11.764 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:11.758+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:12.334 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:12.331+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:12.435 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:12.426+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:12.547 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:12.540+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:12.790 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:12.784+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:13.350 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:13.341+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:13.352 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:13.346+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:13.396 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:13.392+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:13.467 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:13.461+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:13.604 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:13.595+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:13.830 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:13.823+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:14.312 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:14.305+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:14.313 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:14.309+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:14.421 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:14.416+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:14.489 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:14.482+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:14.585 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:14.579+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:14.800 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:14.795+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:15.273 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:15.265+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:15.273 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:15.268+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:15.379 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:15.373+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:15.392 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:15.386+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 18 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:15.478 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:15.472+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:15.576 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:15.569+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:15.838 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:15.833+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:16.295 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:16.290+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:16.298 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:16.294+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:16.410 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:16.399+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 81 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:16.429 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:16.423+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:16.512 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:16.504+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:16.628 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:16.621+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:16.798 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:16.793+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:17.305 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:17.299+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:17.306 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:17.302+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:17.433 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:17.428+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 140 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:17.467 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:17.463+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:17.499 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:17.492+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:17.662 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:17.656+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:17.780 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:17.774+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:18.351 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:18.346+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:18.354 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:18.350+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:18.423 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:18.406+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:18.456 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:18.449+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:18.519 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:18.514+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:18.644 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:18.637+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:18.741 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:18.735+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:19.369 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:19.361+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:19.370 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:19.365+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:19.423 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:19.408+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:19.451 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:19.444+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:19.523 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:19.518+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:19.645 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:19.639+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:19.768 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:19.763+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:20.373 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:20.369+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:20.375 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:20.371+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:20.413 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:20.407+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:20.431 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:20.413+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:20.505 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:20.501+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:20.645 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:20.636+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:20.788 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:20.783+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:21.357 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:21.352+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:21.358 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:21.355+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:21.396 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:21.389+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:21.465 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:21.452+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:21.496 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:21.492+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:21.614 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:21.604+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:21.759 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:21.754+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:22.378 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:22.371+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:22.398 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:22.388+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:22.402 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:22.392+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:22.416 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:22.410+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:22.441 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:22.425+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:22.484 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:22.477+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:22.604 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:22.597+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:22.729 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:22.721+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:23.343 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:23.336+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:23.396 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:23.389+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:23.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:23.392+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:23.428 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:23.420+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:23.460 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:23.444+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:23.493 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:23.488+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:23.559 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:23.552+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:23.739 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:23.732+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:24.325 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:24.319+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:24.403 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:24.395+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:24.404 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:24.398+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:24.489 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:24.482+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:24.504 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:24.484+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:24.515 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:24.500+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:24.611 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:24.603+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:24.785 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:24.776+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:25.329 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:25.322+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:25.401 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:25.393+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:25.402 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:25.396+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:25.453 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:25.446+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:25.508 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:25.482+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:25.521 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:25.503+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:25.685 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:25.675+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:25.813 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:25.805+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:26.336 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:26.330+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:26.416 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:26.411+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:26.450 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:26.443+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:26.452 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:26.447+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:26.545 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:26.537+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:26.579 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:26.562+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:26.695 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:26.688+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:26.835 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:26.829+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:27.373 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:27.366+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:27.420 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:27.414+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:27.490 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:27.482+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:27.490 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:27.486+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:27.611 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:27.588+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:27.624 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:27.606+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:27.667 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:27.660+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:27.807 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:27.802+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:28.363 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:28.356+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:28.463 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:28.456+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:28.477 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:28.469+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:28.478 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:28.473+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:28.576 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:28.569+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:28.605 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:28.587+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:28.681 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:28.672+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:28.853 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:28.849+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:29.416 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:29.409+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:29.512 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:29.504+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:29.513 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:29.508+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:29.515 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:29.510+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:29.593 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:29.586+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:29.623 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:29.607+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:29.672 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:29.649+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:29.889 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:29.882+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:30.435 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:30.428+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:30.507 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:30.501+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:30.515 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:30.505+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:30.537 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:30.532+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:30.600 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:30.593+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:30.630 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:30.612+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:30.657 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:30.648+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:30.905 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:30.901+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:31.444 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:31.437+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:31.500 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:31.494+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:31.501 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:31.496+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:31.530 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:31.524+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:31.603 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:31.581+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:31.617 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:31.599+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:31.697 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:31.690+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:31.892 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:31.886+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:32.407 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:32.400+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:32.481 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:32.475+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:32.482 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:32.478+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:32.520 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:32.516+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:32.618 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:32.597+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:32.632 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:32.614+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:32.714 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:32.707+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:32.853 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:32.848+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:33.371 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:33.363+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:33.503 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:33.498+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:33.530 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:33.524+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:33.531 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:33.526+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:33.674 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:33.646+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:33.690 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:33.667+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:33.706 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:33.699+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:33.894 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:33.889+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:34.413 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:34.406+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:34.476 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:34.471+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:34.525 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:34.519+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:34.527 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:34.522+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:34.705 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:34.698+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:34.717 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:34.698+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:34.732 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:34.712+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:34.856 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:34.851+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:35.439 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:35.431+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:35.491 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:35.486+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:35.522 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:35.515+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:35.523 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:35.518+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:35.741 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:35.733+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:35.759 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:35.752+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:35.791 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:35.773+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:35.816 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:35.808+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:36.450 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:36.445+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:36.479 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:36.473+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:36.523 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:36.516+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:36.524 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:36.520+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:36.774 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:36.768+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:36.818 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:36.813+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:36.834 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:36.814+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:36.851 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:36.830+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:37.459 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:37.454+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:37.463 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:37.458+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:37.535 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:37.528+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:37.537 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:37.532+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:37.797 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:37.790+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:37.826 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:37.802+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:37.838 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:37.821+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:37.863 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:37.858+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:38.458 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:38.454+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:38.507 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:38.500+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:38.510 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:38.504+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:38.511 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:38.507+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:38.802 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:38.797+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:38.813 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:38.783+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:38.821 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:38.808+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:38.859 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:38.854+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:39.474 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:39.467+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:39.481 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:39.474+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:39.496 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:39.488+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:39.496 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:39.491+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:39.648 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:39.640+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:39.804 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:39.782+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:39.818 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:39.800+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:39.858 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:39.850+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:39.902 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:39.895+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:40.466 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:40.461+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:40.498 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:40.491+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:40.505 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:40.499+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:40.507 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:40.502+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:40.634 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:40.626+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:40.833 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:40.828+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:40.841 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:40.835+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:40.861 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:40.851+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:40.907 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:40.901+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:41.443 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:41.437+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:41.523 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:41.517+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:41.555 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:41.548+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:41.555 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:41.551+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:41.623 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:41.616+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:41.814 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:41.807+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:41.891 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:41.885+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:41.913 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:41.890+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:41.927 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:41.909+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:42.299 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:25:42.291+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 41sec e338 v338) 2019-12-09T16:25:42.468 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:42.463+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:42.495 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:42.488+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:42.572 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:42.567+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:42.573 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:42.569+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:42.621 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:42.614+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:42.787 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:42.780+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:42.917 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:42.910+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:42.942 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:42.936+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:42.945 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:42.929+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:43.476 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:43.468+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:43.484 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:43.478+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:43.547 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:43.542+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:43.550 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:43.546+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:43.663 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:43.656+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:43.771 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:43.763+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:43.928 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:43.922+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:43.986 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:43.979+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:44.018 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:44.000+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:44.509 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:44.503+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:44.509 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:44.502+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:44.544 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:44.533+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:44.545 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:44.540+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:44.622 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:44.614+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:44.758 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:44.751+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:44.940 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:44.935+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:45.007 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:45.002+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:45.028 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:45.012+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:45.520 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:45.514+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:45.521 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:45.516+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:45.534 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:45.527+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:45.556 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:45.551+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:45.602 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:45.595+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:45.794 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:45.788+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:45.979 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:45.972+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:46.036 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:46.013+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:46.050 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:46.031+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:46.479 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:46.472+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:46.480 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:46.475+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:46.536 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:46.529+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:46.597 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:46.593+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:46.655 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:46.649+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:46.833 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:46.828+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:47.011 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:47.005+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:47.047 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:47.041+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:47.078 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:47.061+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:47.325 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:25:47.315+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 41sec e338 v338) 2019-12-09T16:25:47.465 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:47.459+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:47.466 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:47.461+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:47.536 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:47.531+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:47.629 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:47.623+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:47.630 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:47.624+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:47.791 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:47.785+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:48.059 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:48.053+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:48.069 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:48.063+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:48.104 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:48.084+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:48.445 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:48.438+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:48.446 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:48.441+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:48.582 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:48.575+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:48.669 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:48.663+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:48.672 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:48.666+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:48.802 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:48.796+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:49.099 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:49.093+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:49.148 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:49.126+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:49.160 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:49.144+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:49.460 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:49.453+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:49.461 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:49.456+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:49.600 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:49.593+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:49.674 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:49.668+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:49.677 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:49.671+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:49.757 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:49.751+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:50.059 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:50.054+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:50.204 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:50.189+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:50.215 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:50.201+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:50.471 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:50.465+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:50.472 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:50.468+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:50.578 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:50.572+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:50.666 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:50.660+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:50.685 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:50.679+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:50.739 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:50.732+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:51.056 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:51.049+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:51.222 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:51.200+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:51.230 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:51.216+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:51.438 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:51.431+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:51.439 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:51.435+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:51.590 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:51.583+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:51.640 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:51.633+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:51.702 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:51.697+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:51.778 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:51.772+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:52.075 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:52.069+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:52.177 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:52.172+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:52.204 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:52.191+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:52.345 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:25:52.338+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 41sec e338 v338) 2019-12-09T16:25:52.405 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:52.399+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:52.405 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:52.401+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:52.630 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:52.625+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:52.669 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:52.664+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:52.703 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:52.698+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:52.797 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:52.792+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:53.120 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:53.114+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:53.191 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:53.167+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:53.205 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:53.186+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:53.393 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:53.387+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:53.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:53.392+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:53.590 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:53.585+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:53.708 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:53.703+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:53.755 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:53.750+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:53.771 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:53.764+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:54.080 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:54.073+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:54.165 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:54.159+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:54.201 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:54.178+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:54.385 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:54.380+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:54.386 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:54.381+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:54.566 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:54.561+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:54.741 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:54.734+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:54.778 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:54.772+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:54.814 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:54.807+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:55.044 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:55.040+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:55.222 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:55.217+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:55.250 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:55.235+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:55.374 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:55.369+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:55.376 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:55.373+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:55.557 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:55.550+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:55.696 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:55.690+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:55.782 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:55.769+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:55.792 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:55.786+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:56.008 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:56.002+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:56.246 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:56.223+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:56.259 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:56.242+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:56.393 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:56.387+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:56.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:56.392+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:56.519 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:56.513+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:56.715 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:56.708+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:56.809 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:56.803+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:56.820 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:56.813+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:57.053 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:57.047+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:57.238 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:57.232+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:57.270 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:57.254+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:57.362 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:25:57.358+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 41sec e338 v338) 2019-12-09T16:25:57.432 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:57.425+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:57.434 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:57.429+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:57.510 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:57.504+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:57.756 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:57.751+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:57.811 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:57.805+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:57.825 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:57.819+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:58.018 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:58.013+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:58.295 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:58.288+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:58.320 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:58.307+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:58.446 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:58.441+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:58.448 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:58.444+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:58.558 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:58.551+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:58.778 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:58.772+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:58.805 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:58.799+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:58.852 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:58.847+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:25:59.015 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:25:59.010+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:25:59.368 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:59.344+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:25:59.389 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:25:59.364+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:25:59.419 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:59.414+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:25:59.423 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:25:59.419+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:25:59.558 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:25:59.551+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:25:59.768 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:25:59.762+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:25:59.797 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:25:59.792+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:25:59.836 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:25:59.829+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:26:00.013 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:26:00.007+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:26:00.341 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:26:00.320+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:26:00.356 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:26:00.338+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:26:00.440 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:26:00.432+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:26:00.442 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:26:00.436+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:26:00.563 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:26:00.557+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:26:00.794 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:26:00.788+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:26:00.829 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:26:00.823+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:26:00.883 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:26:00.878+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:26:00.970 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:26:00.965+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:26:01.354 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:26:01.347+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:26:01.383 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:26:01.366+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:26:01.444 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:26:01.438+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:26:01.449 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:26:01.444+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:26:01.570 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:26:01.563+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:26:01.803 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:26:01.796+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.409013+0000 front 2019-12-09T16:24:28.921405+0000 (oldest deadline 2019-12-09T16:25:09.024274+0000) 2019-12-09T16:26:01.863 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:26:01.856+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.143825+0000 front 2019-12-09T16:24:26.146386+0000 (oldest deadline 2019-12-09T16:25:39.060647+0000) 2019-12-09T16:26:01.869 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:26:01.863+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:29.601699+0000 front 2019-12-09T16:24:28.770277+0000 (oldest deadline 2019-12-09T16:25:12.664084+0000) 2019-12-09T16:26:01.974 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:26:01.968+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.509167+0000 front 2019-12-09T16:24:25.896491+0000 (oldest deadline 2019-12-09T16:25:09.032946+0000) 2019-12-09T16:26:02.355 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:26:02.348+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:24:41.781469+0000 (oldest deadline 2019-12-09T16:25:21.781469+0000) 2019-12-09T16:26:02.387 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:26:02.370+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4355.0:111711 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-09T16:26:02.435 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:26:02.426+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 41sec e338 v338) 2019-12-09T16:26:02.464 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:26:02.458+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:26.767185+0000 front 2019-12-09T16:24:28.569250+0000 (oldest deadline 2019-12-09T16:25:12.637701+0000) 2019-12-09T16:26:02.469 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:26:02.464+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:26:02.596 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:26:02.590+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:24:28.920456+0000 front 2019-12-09T16:24:25.305550+0000 (oldest deadline 2019-12-09T16:25:09.343765+0000) 2019-12-09T16:26:03.463 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:26:03.459+0000 1694f700 -1 osd.0 339 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:26:04.626 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:26:04.556+0000 1694f700 -1 osd.0 340 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118427 5.5 5:a93a5511:::notify.2:head [watch ping cookie 398400336 gen 1] snapc 0=[] ondisk+write+known_if_redirected e334) 2019-12-09T16:26:06.115 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:26:06.075+0000 20e25700 -1 RGWWatcher::handle_error cookie 398400336 err (107) Transport endpoint is not connected 2019-12-09T16:26:37.350 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:26:37.084+0000 1694f700 -1 osd.1 340 get_health_metrics reporting 1 slow ops, oldest is osd_repop(osd.0.0:29313 5.5 e338/331 5:a93a5511:::notify.2:head v 338'245) 2019-12-09T16:27:37.311 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:37.306+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:37.534 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:37.527+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:38.280 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:38.272+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:38.591 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:38.583+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:39.127 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:39.122+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:39.295 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:39.289+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:39.322 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:39.317+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:39.639 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:39.633+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:40.030 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:40.025+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:40.154 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:40.143+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:40.276 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:40.271+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:40.298 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:40.293+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:40.531 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:40.525+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:40.693 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:40.687+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:41.005 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:40.999+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:41.129 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:41.124+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:41.261 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:41.255+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:41.323 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:41.318+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:41.522 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:41.517+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:41.713 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:41.707+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:42.015 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:42.008+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:42.139 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:42.134+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:42.227 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:42.222+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:42.376 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:42.371+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:42.557 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:42.551+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:42.679 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:42.674+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:43.037 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:43.031+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:43.187 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:43.180+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:43.193 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:43.188+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:43.423 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:43.418+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:43.545 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:43.539+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:43.669 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:43.663+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:43.985 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:43.979+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:44.069 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:44.064+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:44.168 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:44.163+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:44.213 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:44.201+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:44.388 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:44.382+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:44.554 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:44.548+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:44.646 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:44.640+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:44.976 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:44.970+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:45.047 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:45.041+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:45.213 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:45.209+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:45.242 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:45.236+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:45.355 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:45.350+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:45.587 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:45.578+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:45.664 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:45.658+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:45.999 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:45.994+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:46.097 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:46.092+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:46.202 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:46.197+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:46.246 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:46.240+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:46.406 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:46.401+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:46.592 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:46.586+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:46.653 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:46.646+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:47.054 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:47.048+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:47.126 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:47.118+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:47.196 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:47.190+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:47.262 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:47.254+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:47.421 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:47.416+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:47.627 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:47.621+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:47.699 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:47.693+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:48.024 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:48.017+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:48.085 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:48.079+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:48.170 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:48.164+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:48.281 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:48.274+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:48.461 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:48.455+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:48.584 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:48.578+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:48.735 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:48.728+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:48.978 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:48.972+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:49.112 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:49.106+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:49.185 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:49.178+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:49.327 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:49.320+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:49.430 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:49.424+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:49.559 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:49.553+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:49.720 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:49.714+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:49.998 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:49.992+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:50.125 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:50.119+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:50.147 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:50.141+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:50.323 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:50.317+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:50.484 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:50.478+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:50.612 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:50.605+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:50.704 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:50.698+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:50.965 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:50.959+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:51.116 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:51.110+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:51.160 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:51.154+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:51.356 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:51.350+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:51.500 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:51.495+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:51.610 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:51.604+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:51.679 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:51.673+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:51.948 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:51.944+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:52.152 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:52.148+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:52.206 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:52.201+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:52.404 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:52.398+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:52.475 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:52.470+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:52.625 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:52.620+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:52.670 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:52.664+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:52.925 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:52.920+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:53.181 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:53.175+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:53.212 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:53.207+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:53.381 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:53.376+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:53.475 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:53.469+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:53.595 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:53.589+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:53.676 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:53.670+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:53.968 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:53.962+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:54.205 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:54.201+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:54.239 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:54.234+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:54.434 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:54.429+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:54.488 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:54.482+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:54.621 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:54.615+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:54.644 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:54.638+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:54.988 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:54.983+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:55.170 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:55.164+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:55.274 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:55.269+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:55.487 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:55.480+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:55.527 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:55.522+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:55.601 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:55.595+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:55.605 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:55.597+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:55.947 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:55.942+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:56.140 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:56.134+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:56.286 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:56.281+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:56.496 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:56.490+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:56.541 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:56.536+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:56.558 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:56.552+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:56.582 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:56.576+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:56.912 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:56.907+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:57.162 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:57.157+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:57.341 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:57.335+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:57.521 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:57.515+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:57.527 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:57.521+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:57.595 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:57.590+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:57.596 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:57.591+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:57.940 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:57.935+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:58.116 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:58.111+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:58.309 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:58.299+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:58.515 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:58.510+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:58.546 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:58.540+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:58.570 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:58.564+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:58.583 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:58.577+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:58.942 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:58.938+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:27:59.146 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:27:59.140+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:27:59.324 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:27:59.319+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:27:59.489 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:27:59.483+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:27:59.525 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:27:59.519+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:27:59.617 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:27:59.612+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:27:59.634 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:27:59.628+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:27:59.941 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:27:59.936+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:00.192 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:00.188+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:00.322 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:00.316+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:00.499 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:00.493+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:00.533 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:00.527+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:00.617 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:00.612+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:00.652 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:00.640+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:00.952 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:00.947+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:01.185 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:01.181+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:01.284 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:01.278+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:01.532 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:01.527+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:01.565 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:01.559+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:01.594 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:01.590+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:01.691 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:01.683+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:01.911 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:01.906+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:02.198 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:02.193+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:02.266 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:02.261+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:02.520 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:02.515+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:02.550 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:02.546+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:02.582 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:02.577+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:02.691 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:02.686+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:02.920 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:02.915+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:03.176 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:03.170+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:03.284 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:03.278+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:03.543 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:03.537+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:03.553 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:03.546+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:03.553 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:03.548+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:03.702 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:03.693+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:03.945 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:03.940+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:04.191 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:04.186+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:04.296 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:04.290+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:04.538 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:04.533+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:04.544 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:04.538+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:04.591 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:04.584+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:04.669 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:04.663+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:04.925 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:04.919+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:05.212 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:05.199+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:05.288 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:05.283+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:05.523 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:05.517+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:05.587 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:05.581+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:05.587 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:05.581+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:05.640 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:05.634+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:05.942 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:05.937+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:06.178 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:06.173+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:06.251 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:06.246+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:06.500 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:06.496+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:06.611 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:06.605+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:06.618 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:06.611+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:06.635 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:06.628+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:06.995 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:06.989+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:07.203 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:07.198+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:07.246 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:07.240+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:07.484 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:07.479+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:07.632 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:07.626+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:07.673 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:07.662+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:07.674 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:07.668+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:07.965 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:07.960+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:08.008 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:28:08.003+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 43sec e344 v344) 2019-12-09T16:28:08.165 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:08.160+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:08.206 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:08.201+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:08.480 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:08.474+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:08.624 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:08.618+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:08.639 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:08.628+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:08.707 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:08.702+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:08.930 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:08.925+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:09.177 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:09.172+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:09.211 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:09.206+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:09.474 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:09.468+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:09.619 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:09.613+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:09.652 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:09.646+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:09.758 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:09.752+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:09.934 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:09.929+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:10.149 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:10.143+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:10.204 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:10.199+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:10.445 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:10.440+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:10.604 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:10.599+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:10.630 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:10.625+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:10.803 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:10.796+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:10.947 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:10.941+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:11.165 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:11.159+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:11.181 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:11.176+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:11.480 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:11.476+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:11.592 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:11.586+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:11.662 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:11.656+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:11.784 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:11.778+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:11.909 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:11.904+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:12.151 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:12.145+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:12.194 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:12.189+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:12.467 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:12.460+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:12.622 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:12.617+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:12.712 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:12.705+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:12.762 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:12.755+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:12.948 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:12.943+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:13.015 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:28:13.011+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 43sec e344 v344) 2019-12-09T16:28:13.155 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:13.150+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:13.156 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:13.152+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:13.491 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:13.485+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:13.579 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:13.574+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:13.759 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:13.753+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:13.810 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:13.803+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:13.924 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:13.919+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:14.166 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:14.161+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:14.179 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:14.173+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:14.478 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:14.472+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:14.540 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:14.535+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:14.771 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:14.760+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:14.848 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:14.842+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:14.903 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:14.897+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:15.139 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:15.133+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:15.181 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:15.176+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:15.460 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:15.455+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:15.499 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:15.492+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:15.817 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:15.812+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:15.838 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:15.833+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:15.910 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:15.905+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:16.140 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:16.134+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:16.177 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:16.171+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:16.433 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:16.427+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:16.542 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:16.536+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:16.814 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:16.808+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:16.827 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:16.822+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:16.960 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:16.954+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:17.139 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:17.135+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:17.146 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:15:53.411 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:17.152 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:15:53.417 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:17.163 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:17.158+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:17.165 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:15:53.387 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:17.166 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:15:53.388 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:17.170 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:15:53.392 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:17.171 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:15:53.392 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:17.405 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:17.400+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:17.422 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:15:53.687 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:17.509 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:17.504+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:17.512 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:15:53.734 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:17.517 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:15:53.739 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:17.834 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:17.830+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:17.856 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:17.851+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:18.007 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:18.000+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:18.095 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:18.090+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:18.120 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:18.115+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:18.182 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:28:18.168+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 43sec e344 v344) 2019-12-09T16:28:18.375 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:18.369+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:18.520 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:18.516+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:18.837 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:18.832+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:18.869 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:18.864+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:18.990 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:18.985+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:19.091 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:19.087+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:19.092 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:19.087+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:19.414 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:19.410+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:19.535 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:19.530+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:19.815 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:19.806+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:19.844 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:19.839+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:19.956 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:19.951+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:20.079 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:20.074+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:20.143 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:20.137+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:20.450 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:20.445+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:20.575 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:20.571+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:20.841 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:20.836+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:20.853 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:20.848+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:20.943 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:20.936+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:21.079 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:21.074+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:21.166 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:21.161+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:21.490 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:21.484+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:21.557 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:21.554+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:21.798 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:21.793+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:21.813 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:21.808+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:21.970 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:21.965+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:22.077 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:22.072+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:22.188 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:22.183+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:22.449 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:22.444+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:22.550 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:28:22.545+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.212666+0000 front 2019-12-09T16:27:01.166776+0000 (oldest deadline 2019-12-09T16:27:40.197604+0000) 2019-12-09T16:28:22.842 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:28:22.836+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.546101+0000 front 2019-12-09T16:26:53.542257+0000 (oldest deadline 2019-12-09T16:27:37.115066+0000) 2019-12-09T16:28:22.843 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:28:22.837+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:01.272139+0000 front 2019-12-09T16:26:57.480250+0000 (oldest deadline 2019-12-09T16:27:39.134752+0000) 2019-12-09T16:28:22.964 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:28:22.959+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:59.549736+0000 front 2019-12-09T16:26:57.452647+0000 (oldest deadline 2019-12-09T16:27:43.244446+0000) 2019-12-09T16:28:23.117 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:28:23.112+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:55.812953+0000 front 2019-12-09T16:27:01.238381+0000 (oldest deadline 2019-12-09T16:27:39.151434+0000) 2019-12-09T16:28:23.230 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:28:23.225+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:27:00.988966+0000 front 2019-12-09T16:26:59.128367+0000 (oldest deadline 2019-12-09T16:27:39.084870+0000) 2019-12-09T16:28:23.253 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:28:23.250+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 43sec e344 v344) 2019-12-09T16:28:23.411 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:28:23.406+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:26:56.674176+0000 front 2019-12-09T16:26:59.225049+0000 (oldest deadline 2019-12-09T16:27:36.804477+0000) 2019-12-09T16:28:43.559 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:16:19.823 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:43.565 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:16:19.830 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:43.614 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:16:19.836 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:43.630 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:16:19.851 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:43.638 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:16:19.903 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:43.656 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:16:19.878 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:43.660 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:16:19.880 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:43.711 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:16:19.933 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:43.729 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:16:19.951 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:29:21.406 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:21.400+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:22.362 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:22.356+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:22.470 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:22.465+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:23.335 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:23.329+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:23.483 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:23.479+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:23.491 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:23.484+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:23.534 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:23.529+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:24.242 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:24.234+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:24.389 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:24.382+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:24.503 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:24.495+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:24.526 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:24.521+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:24.578 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:24.571+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:24.582 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:24.577+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:25.292 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:25.284+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:25.418 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:25.412+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:25.483 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:25.476+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:25.550 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:25.542+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:25.584 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:25.576+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:25.586 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:25.579+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:26.308 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:26.300+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:26.406 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:26.399+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:26.439 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:26.432+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:26.461 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:26.453+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:26.547 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:26.540+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:26.548 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:26.542+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:26.555 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:26.548+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:27.284 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:27.277+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:27.373 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:27.366+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:27.448 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:27.441+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:27.481 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:27.473+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:27.510 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:27.503+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:27.511 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:27.506+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:27.514 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:27.507+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:28.318 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:28.311+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:28.352 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:28.343+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:28.476 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:28.469+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:28.502 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:28.496+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:28.502 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:28.494+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:28.538 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:28.533+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:28.542 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:28.538+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:29.345 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:29.338+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:29.399 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:29.393+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:29.513 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:29.506+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:29.521 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:29.514+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:29.528 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:29.521+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:29.528 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:29.523+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:29.540 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:29.533+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:30.371 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:30.364+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:30.433 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:30.427+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:30.543 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:30.535+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:30.575 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:30.568+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:30.579 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:30.572+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:30.580 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:30.576+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:30.587 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:30.581+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:31.373 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:31.366+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:31.394 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:31.388+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:31.544 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:31.537+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:31.566 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:31.561+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:31.568 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:31.564+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:31.581 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:31.575+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:31.630 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:31.623+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:32.368 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:32.362+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:32.405 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:32.398+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:32.579 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:32.572+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:32.580 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:32.575+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:32.589 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:32.583+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:32.602 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:32.593+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:32.606 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:32.598+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:33.336 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:33.330+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:33.421 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:33.416+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:33.564 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:33.558+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:33.583 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:33.578+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:33.610 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:33.604+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:33.631 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:33.623+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:33.632 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:33.627+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:34.300 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:34.295+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:34.397 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:34.391+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:34.580 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:34.575+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:34.595 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:34.590+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:34.596 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:34.592+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:34.618 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:34.612+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:34.627 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:34.621+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:35.283 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:35.276+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:35.391 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:35.385+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:35.543 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:35.538+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:35.604 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:35.599+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:35.605 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:35.601+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:35.649 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:35.643+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:35.665 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:35.659+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:36.320 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:36.314+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:36.398 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:36.393+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:36.533 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:36.528+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:36.562 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:36.556+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:36.563 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:36.558+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:36.680 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:36.674+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:36.689 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:36.685+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:37.331 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:37.325+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:37.408 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:37.401+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:37.549 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:37.542+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:37.609 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:37.602+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:37.610 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:37.606+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:37.658 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:37.653+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:37.684 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:37.679+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:38.384 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:38.379+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:38.407 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:38.401+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:38.579 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:38.572+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:38.585 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:38.579+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:38.591 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:38.585+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:38.691 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:38.685+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:38.725 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:38.714+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:39.438 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:39.432+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:39.463 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:39.458+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:39.599 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:39.594+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:39.602 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:39.598+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:39.628 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:39.622+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:39.703 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:39.697+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:39.727 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:39.721+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:40.416 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:40.408+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:40.448 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:40.442+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:40.597 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:40.592+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:40.645 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:40.640+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:40.646 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:40.642+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:40.759 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:40.748+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:40.766 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:40.761+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:41.467 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:41.462+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:41.483 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:41.478+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:41.552 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:41.546+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:41.652 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:41.645+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:41.653 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:41.648+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:41.762 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:41.757+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:41.804 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:41.797+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:42.500 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:42.495+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:42.501 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:42.494+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:42.522 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:42.516+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:42.656 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:42.651+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:42.657 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:42.653+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:42.762 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:42.755+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:42.854 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:42.848+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:43.533 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:43.527+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:43.547 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:43.540+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:43.548 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:43.544+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:43.640 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:43.634+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:43.644 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:43.640+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:43.800 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:43.794+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:43.888 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:43.883+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:44.510 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:44.504+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:44.544 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:44.538+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:44.551 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:44.545+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:44.686 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:44.680+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:44.690 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:44.686+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:44.832 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:44.826+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:44.880 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:44.874+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:45.525 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:45.519+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:45.528 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:45.522+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:45.586 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:45.580+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:45.662 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:45.656+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:45.663 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:45.658+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:45.849 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:45.843+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:45.933 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:45.926+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:46.544 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:46.537+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:46.583 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:46.576+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:46.589 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:46.583+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:46.684 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:46.677+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:46.685 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:46.681+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:46.838 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:46.827+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:46.918 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:46.910+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:47.515 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:47.509+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:47.579 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:47.572+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:47.625 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:47.621+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:47.696 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:47.690+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:47.700 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:47.695+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:47.851 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:47.845+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:47.888 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:47.882+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:47.997 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:47.992+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:48.532 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:48.526+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:48.613 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:48.607+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:48.617 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:48.609+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:48.690 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:48.684+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:48.695 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:48.691+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:48.846 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:48.840+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:48.944 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:48.934+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:49.056 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:49.045+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 68 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:49.574 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:49.568+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:49.628 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:49.623+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:49.640 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:49.633+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:49.706 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:49.699+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:49.710 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:49.706+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:49.832 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:49.824+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:49.955 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:49.945+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:50.094 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:50.079+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:50.588 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:50.583+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:50.621 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:50.616+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:50.689 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:50.684+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:50.690 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:50.686+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:50.705 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:50.698+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:50.844 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:50.834+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:50.951 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:50.944+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:51.078 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:51.067+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:51.627 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:51.617+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:51.638 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:51.632+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:51.721 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:51.714+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:51.722 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:51.718+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:51.727 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:51.720+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:51.836 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:51.829+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:51.921 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:51.914+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:52.109 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:52.091+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:52.658 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:52.652+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:52.668 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:52.662+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:52.711 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:52.705+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:52.712 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:52.708+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:52.736 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:52.726+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:52.884 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:52.877+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:52.912 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:52.906+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:53.171 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:53.159+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:53.622 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:53.617+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:53.672 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:53.666+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:53.705 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:53.699+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:53.749 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:53.744+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:53.750 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:29:53.740+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e350 v350) 2019-12-09T16:29:53.753 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:53.749+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:53.907 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:53.900+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:53.943 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:53.935+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:54.204 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:54.196+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:29:54.233 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:54.217+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:54.673 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:54.665+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:54.690 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:54.685+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:54.713 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:54.706+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:54.755 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:54.746+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:54.756 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:54.750+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:54.931 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:54.924+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:54.959 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:54.952+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:55.180 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:55.174+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:29:55.206 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:55.194+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:55.661 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:55.656+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:55.662 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:55.652+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:55.737 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:55.731+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:55.766 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:55.760+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:55.767 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:55.763+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:55.933 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:55.919+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:55.980 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:55.973+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:56.207 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:56.185+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:29:56.220 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:56.203+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:56.611 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:56.606+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:56.651 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:56.646+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:56.703 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:56.695+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:56.740 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:56.732+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:56.741 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:56.736+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:56.910 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:56.903+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:57.026 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:57.018+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:57.195 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:57.176+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:29:57.208 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:57.190+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:57.635 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:57.629+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:57.685 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:57.678+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:57.711 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:57.704+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:57.725 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:57.718+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:57.726 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:57.721+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:57.918 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:57.911+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:57.993 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:57.985+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:58.201 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:58.170+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:29:58.215 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:58.196+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:58.662 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:58.655+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:58.693 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:58.687+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:58.727 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:58.720+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:58.755 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:58.749+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:58.761 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:58.756+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:58.766 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:29:58.762+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e350 v350) 2019-12-09T16:29:58.937 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:58.930+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:58.981 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:58.974+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:29:59.172 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:59.165+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:29:59.203 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:29:59.185+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:29:59.681 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:29:59.675+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:29:59.691 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:29:59.686+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:29:59.747 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:59.742+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:29:59.748 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:29:59.744+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:29:59.771 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:29:59.764+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:29:59.956 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:29:59.949+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:29:59.971 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:29:59.964+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:00.224 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:00.201+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:00.236 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:00.219+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:00.696 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:00.689+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:00.724 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:00.717+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:00.741 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:00.733+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:00.791 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:00.787+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:00.794 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:00.790+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:00.941 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:00.933+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:00.964 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:00.954+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:01.230 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:01.212+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:01.240 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:01.226+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:01.664 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:01.657+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:01.689 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:01.684+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:01.697 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:01.690+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:01.846 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:01.840+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:01.850 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:01.845+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:01.938 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:01.931+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:01.980 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:01.972+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:02.289 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:02.265+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:02.301 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:02.284+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:02.653 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:02.647+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:02.716 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:02.700+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:02.717 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:02.710+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:02.864 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:02.857+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:02.868 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:02.863+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:02.921 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:02.913+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:03.033 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:03.026+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:03.365 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:03.336+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:03.373 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:03.354+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:03.622 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:03.616+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:03.667 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:03.659+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:03.763 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:03.757+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:03.796 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:30:03.785+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e350 v350) 2019-12-09T16:30:03.884 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:03.876+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:03.896 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:03.888+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:03.897 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:03.893+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:04.080 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:04.073+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:04.375 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:04.369+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:04.406 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:04.388+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:04.659 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:04.655+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:04.693 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:04.686+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:04.725 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:04.720+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:04.853 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:04.846+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:04.935 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:04.928+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:04.935 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:04.931+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:05.051 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:05.044+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:05.370 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:05.347+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:05.384 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:05.366+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:05.662 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:05.655+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:05.708 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:05.701+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:05.734 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:05.729+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:05.836 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:05.828+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:05.967 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:05.959+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:05.968 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:05.964+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:06.025 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:06.018+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:06.351 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:06.346+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:06.370 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:06.356+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:06.684 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:06.678+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:06.709 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:06.703+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:06.751 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:06.746+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:06.867 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:06.860+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:06.977 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:06.968+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:06.979 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:06.972+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:07.035 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:07.028+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:07.390 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:07.384+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:07.423 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:07.404+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:07.676 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:07.668+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:07.705 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:07.700+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:07.731 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:07.724+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:07.911 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:07.904+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:07.935 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:07.929+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:07.935 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:07.931+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:08.058 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:08.051+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:08.421 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:08.400+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:08.433 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:08.416+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:08.732 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:08.726+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:08.741 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:08.735+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:08.748 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:08.743+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:08.811 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:30:08.800+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e350 v350) 2019-12-09T16:30:08.912 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:08.908+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:08.914 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:08.910+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:08.956 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:08.948+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:09.064 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:09.058+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:09.457 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:09.451+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:09.491 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:09.474+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:09.700 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:09.694+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:09.793 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:09.788+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:09.804 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:09.797+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:09.939 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:09.933+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:09.941 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:09.936+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:10.002 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:09.994+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:10.082 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:10.074+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:10.452 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:10.430+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:10.466 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:10.447+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:10.675 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:10.669+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:10.778 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:10.771+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:10.854 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:10.847+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:10.930 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:10.925+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:10.931 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:10.927+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:11.022 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:11.015+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:11.049 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:11.042+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:11.490 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:11.462+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:11.500 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:11.480+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:11.653 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:11.647+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:11.816 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:11.809+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:11.823 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:11.817+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:11.936 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:11.931+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:11.940 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:11.935+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:12.011 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:12.005+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:12.025 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:12.018+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:12.498 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:12.491+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:12.527 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:12.510+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:12.693 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:12.687+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:12.829 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:12.822+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:12.830 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:12.825+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:12.933 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:12.926+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:12.934 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:12.929+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:12.983 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:12.976+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:13.060 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:13.053+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:13.519 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:13.496+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:13.532 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:13.514+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:13.690 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:13.683+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:13.837 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:30:13.826+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e350 v350) 2019-12-09T16:30:13.845 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:13.838+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:13.846 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:13.841+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:13.904 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:13.897+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:13.917 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:13.903+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:14.030 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:14.023+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:14.039 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:14.032+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:14.492 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:14.485+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:14.526 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:14.508+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:14.744 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:14.737+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:14.875 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:14.869+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:14.894 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:14.886+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:14.957 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:14.949+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:14.959 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:14.952+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:15.009 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:15.002+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:15.025 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:15.019+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:15.496 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:15.473+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:15.510 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:15.491+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:15.750 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:15.743+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:15.856 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:15.848+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:15.909 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:15.902+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:15.973 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:15.965+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:15.975 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:15.969+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:15.984 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:15.977+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:16.046 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:16.038+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:16.554 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:16.531+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:16.568 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:16.550+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:16.719 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:16.712+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:16.861 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:16.848+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:16.938 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:16.932+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:16.967 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:16.961+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:16.995 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:16.989+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:16.998 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:16.992+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:17.023 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:17.016+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:17.552 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:17.527+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:17.566 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:17.547+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:17.698 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:17.692+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:17.892 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:17.886+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:17.905 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:17.900+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:17.953 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:17.945+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:17.954 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:17.950+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:17.979 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:17.972+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:17.984 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:17.977+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:18.527 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:18.509+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:18.540 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:18.522+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:18.684 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:18.679+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:18.856 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:18.849+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:18.918 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:18.913+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:18.929 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:18.921+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:18.930 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:18.926+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:18.965 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:30:18.857+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e350 v350) 2019-12-09T16:30:19.022 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:19.015+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:19.032 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:19.025+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:19.568 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:19.561+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:19.599 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:19.581+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:19.668 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:19.662+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:19.888 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:19.882+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:19.889 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:19.882+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:19.895 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:19.890+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:19.895 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:19.889+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:20.020 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:20.013+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:20.047 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:20.039+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:20.577 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:20.549+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:20.593 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:20.571+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:20.709 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:20.701+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:20.859 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:20.851+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:20.864 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:20.859+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:20.874 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:20.868+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:20.887 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:20.879+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:21.004 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:20.998+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:21.010 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:21.004+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:21.617 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:21.594+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:21.631 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:21.613+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:21.744 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:21.739+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:21.866 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:21.859+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:21.868 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:21.862+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:21.894 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:21.888+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:21.909 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:21.902+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:22.035 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:22.028+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:22.063 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:22.052+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:22.660 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:22.653+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:22.691 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:22.674+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:22.708 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:22.702+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:22.873 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:22.869+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:22.916 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:22.909+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:22.918 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:22.912+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:22.927 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:22.920+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:23.060 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:23.053+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:23.114 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:23.106+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:23.689 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:23.681+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:23.712 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:23.706+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:23.725 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:23.708+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:23.891 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:30:23.881+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e350 v350) 2019-12-09T16:30:23.899 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:23.894+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:23.900 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:23.896+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:23.921 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:23.916+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:23.956 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:23.948+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:24.038 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:24.030+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:24.109 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:24.101+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:24.695 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:24.690+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:24.741 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:24.735+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:24.759 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:24.747+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:24.887 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:24.883+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:24.889 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:24.885+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:24.936 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:24.929+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:24.951 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:24.946+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:25.042 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:25.035+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:25.063 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:25.057+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:25.681 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:25.674+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:25.734 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:25.702+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:25.743 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:25.730+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:25.869 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:25.861+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:25.870 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:25.865+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:25.893 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:25.885+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:25.991 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:25.986+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:26.060 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:26.053+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:26.090 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:26.082+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:26.684 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:26.678+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:26.760 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:26.736+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:26.772 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:26.755+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:26.861 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:26.856+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:26.865 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:26.861+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:26.912 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:26.902+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:26.951 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:26.944+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:27.044 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:27.037+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:27.099 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:27.092+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:27.724 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:27.717+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:27.733 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:27.726+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:27.763 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:27.745+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:27.823 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:27.815+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:27.824 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:27.819+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:27.924 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:27.917+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:27.975 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:27.970+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:28.042 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:28.035+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:28.117 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:28.110+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:28.746 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:28.723+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:28.759 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:28.742+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:28.776 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:28.770+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:28.860 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:28.850+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:28.861 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:28.856+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:28.931 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:28.924+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:29.007 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:29.002+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:29.022 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:30:29.011+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e350 v350) 2019-12-09T16:30:29.052 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:29.045+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:29.126 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:29.118+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:29.793 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:29.787+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:29.819 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:29.786+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:29.839 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:29.806+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:29.868 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:29.857+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:29.872 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:29.862+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:29.987 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:29.979+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:30.054 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:30.047+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:30.066 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:30.059+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:30.107 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:30.100+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:30.836 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:30.831+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:30.881 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:30.858+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:30.888 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:30.881+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:30.890 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:30.884+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:30.893 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:30.876+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:31.016 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:31.008+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:31.067 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:31.060+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:31.069 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:31.064+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:31.095 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:31.088+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:31.840 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:31.836+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:31.844 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:31.839+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:31.889 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:31.883+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:31.910 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:31.893+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:31.927 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:31.906+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:31.970 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:31.964+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:32.107 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:32.099+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:32.121 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:32.116+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:32.141 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:32.134+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:32.886 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:32.881+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:32.891 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:32.886+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:32.913 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:32.901+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:32.917 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:32.893+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:32.931 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:32.912+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:32.942 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:32.935+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:33.126 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:33.119+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:33.127 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:33.120+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:33.182 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:33.174+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:33.858 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:33.852+0000 1694f700 -1 osd.0 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:39.024203+0000 front 2019-12-09T16:28:39.238872+0000 (oldest deadline 2019-12-09T16:29:24.058022+0000) 2019-12-09T16:30:33.868 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:33.855+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:33.882 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:33.875+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:33.900 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:33.894+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:33.913 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:33.895+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:33.985 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:33.978+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:34.038 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:30:34.033+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e350 v350) 2019-12-09T16:30:34.118 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:34.111+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:34.168 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:34.162+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:34.179 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:34.173+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:34.872 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:34.866+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:34.884 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:34.879+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:34.964 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:34.940+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:34.967 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:34.959+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:34.976 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:34.959+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:35.131 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:35.123+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:35.143 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:35.137+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:35.156 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:35.150+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:35.841 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:35.836+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:35.849 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:35.844+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:36.002 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:35.994+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:36.019 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:35.995+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:36.032 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:36.014+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:36.102 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:36.095+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:36.149 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:36.142+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:36.171 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:36.165+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:36.808 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:36.802+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:36.850 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:36.845+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:36.990 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:36.984+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:37.057 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:37.033+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:37.070 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:37.052+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:37.108 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:37.101+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:37.152 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:37.145+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:37.208 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:37.204+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:37.796 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:37.789+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:37.819 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:37.813+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:38.035 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:38.028+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:38.090 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:38.074+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:38.116 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:38.093+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:38.154 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:38.145+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:38.161 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:38.154+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:38.198 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:38.192+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:38.798 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:30:38.793+0000 1694f700 -1 osd.2 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.595674+0000 front 2019-12-09T16:28:40.470665+0000 (oldest deadline 2019-12-09T16:29:22.715353+0000) 2019-12-09T16:30:38.811 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:38.802+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:39.060 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:30:39.053+0000 1694f700 -1 osd.5 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:38.674556+0000 front 2019-12-09T16:28:38.688765+0000 (oldest deadline 2019-12-09T16:29:23.935708+0000) 2019-12-09T16:30:39.085 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:39.070+0000 1694f700 -1 osd.6 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 ever on either front or back, first ping sent 2019-12-09T16:29:14.188193+0000 (oldest deadline 2019-12-09T16:29:54.188193+0000) 2019-12-09T16:30:39.093 INFO:tasks.ceph.osd.6.smithi022.stderr:2019-12-09T16:30:39.080+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 160 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-12-09T16:30:39.120 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:30:39.113+0000 1694f700 -1 osd.7 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:36.422207+0000 front 2019-12-09T16:28:36.419052+0000 (oldest deadline 2019-12-09T16:29:21.205513+0000) 2019-12-09T16:30:39.146 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:30:39.133+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 45sec e350 v350) 2019-12-09T16:30:39.171 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:30:39.166+0000 1694f700 -1 osd.3 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:40.739206+0000 front 2019-12-09T16:28:40.868752+0000 (oldest deadline 2019-12-09T16:29:26.037090+0000) 2019-12-09T16:30:39.189 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:30:39.183+0000 1694f700 -1 osd.4 350 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:28:41.103015+0000 front 2019-12-09T16:28:39.968684+0000 (oldest deadline 2019-12-09T16:29:21.975236+0000) 2019-12-09T16:30:39.890 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:39.849+0000 1694f700 -1 osd.0 351 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:40.967 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:30:40.902+0000 1694f700 -1 osd.0 352 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4355.0:118829 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3474105104 gen 1] snapc 0=[] ondisk+write+known_if_redirected e346) 2019-12-09T16:30:42.242 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:30:42.215+0000 20e25700 -1 RGWWatcher::handle_error cookie 3474105104 err (107) Transport endpoint is not connected 2019-12-09T16:30:47.221 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:30:47.196+0000 1694f700 -1 osd.1 351 get_health_metrics reporting 1 slow ops, oldest is osd_repop(osd.0.0:29319 5.5 e348/342 5:a93a5511:::notify.2:head v 348'250) 2019-12-09T16:31:51.081 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:31:51.072+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:31:51.383 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:31:51.377+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:31:51.585 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:31:51.578+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:31:51.656 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:31:51.649+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:31:51.865 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:31:51.858+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:31:52.076 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:31:52.070+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:31:52.402 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:31:52.396+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:31:52.589 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:31:52.582+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:31:52.621 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:31:52.613+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:31:52.886 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:31:52.879+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:31:53.070 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:31:53.063+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:31:53.099 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:31:53.093+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:31:53.384 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:31:53.379+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:31:53.617 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:31:53.606+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:31:53.659 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:31:53.653+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:31:53.871 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:31:53.865+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:31:54.125 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:31:54.118+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:31:54.134 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:31:54.129+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:31:54.384 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:31:54.379+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:31:54.592 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:31:54.587+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:31:54.693 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:31:54.684+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:31:54.760 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:31:54.754+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:31:54.838 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:31:54.832+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:31:55.169 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:31:55.161+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:31:55.176 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:31:55.167+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:31:55.381 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:31:55.374+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:31:55.599 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:31:55.594+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:31:55.664 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:31:55.658+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:31:55.746 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:31:55.740+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:31:55.807 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:31:55.801+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:31:56.180 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:31:56.174+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:31:56.192 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:31:56.186+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:31:56.400 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:31:56.395+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:31:56.615 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:31:56.610+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:31:56.619 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:31:56.613+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:31:56.778 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:31:56.772+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:31:56.825 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:31:56.817+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:31:57.202 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:31:57.196+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:31:57.224 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:31:57.217+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:31:57.382 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:31:57.377+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:31:57.588 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:31:57.584+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:31:57.597 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:31:57.592+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:31:57.799 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:31:57.793+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:31:57.837 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:31:57.833+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:31:58.209 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:31:58.201+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:31:58.226 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:31:58.220+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:31:58.423 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:31:58.418+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:31:58.583 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:31:58.578+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:31:58.589 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:31:58.584+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:31:58.757 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:31:58.752+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:31:58.791 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:31:58.786+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:31:59.219 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:31:59.213+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:31:59.220 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:31:59.214+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:31:59.447 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:31:59.441+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:31:59.587 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:31:59.583+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:31:59.612 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:31:59.607+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:31:59.750 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:31:59.743+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:31:59.774 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:31:59.768+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:00.214 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:00.208+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:00.220 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:00.215+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:00.485 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:00.480+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:00.561 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:00.556+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:00.628 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:00.623+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:00.716 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:00.709+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:00.783 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:00.778+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:01.181 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:01.175+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:01.191 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:01.185+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:01.457 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:01.452+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:01.606 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:01.601+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:01.649 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:01.644+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:01.762 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:01.756+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:01.765 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:01.759+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:02.151 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:02.145+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:02.212 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:02.206+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:02.412 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:02.407+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:02.659 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:02.645+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:02.680 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:02.676+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:02.720 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:02.714+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:02.744 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:02.739+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:03.138 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:03.132+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:03.189 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:03.183+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:03.415 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:03.410+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:03.678 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:03.672+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:03.695 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:03.686+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:03.699 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:03.694+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:03.715 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:03.710+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:04.189 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:04.183+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:04.215 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:04.210+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:04.429 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:04.424+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:04.650 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:04.645+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:04.688 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:04.683+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:04.688 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:04.683+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:04.761 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:04.757+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:04.896 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:19:41.160 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:04.966 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:19:41.230 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:04.976 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:19:41.198 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:04.978 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:19:41.200 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:04.987 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:19:41.209 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:05.001 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:19:41.223 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:05.182 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:05.177+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:05.214 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:19:41.436 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:05.241 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:19:41.463 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:05.249 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:05.244+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:05.428 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:05.423+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:05.447 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:19:41.711 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:05.635 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:05.631+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:05.712 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:05.707+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:05.732 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:05.727+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:05.748 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:05.743+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:06.177 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:06.173+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:06.219 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:06.214+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:06.385 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:06.379+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:06.609 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:06.603+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:06.707 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:06.701+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:06.728 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:06.724+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:06.765 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:06.760+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:07.006 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:19:43.227 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:07.007 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:19:43.229 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:07.031 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:19:43.253 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:07.039 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:19:43.261 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:07.041 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:19:43.263 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:07.052 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:19:43.274 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:07.176 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:07.172+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:07.228 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:07.223+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:07.315 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:19:43.579 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:07.316 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:19:43.581 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:07.332 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:19:43.596 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:32:07.432 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:07.427+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:07.580 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:07.576+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:07.666 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:07.661+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:07.720 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:07.715+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:07.793 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:07.789+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:08.200 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:08.196+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:08.239 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:08.234+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:08.472 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:08.468+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:08.613 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:08.608+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:08.627 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:08.622+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:08.741 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:08.736+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:08.779 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:08.775+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:09.236 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:09.232+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:09.286 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:09.280+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:09.485 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:09.480+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:09.600 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:09.594+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:09.636 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:09.631+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:09.781 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:09.773+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:09.814 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:09.809+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:10.267 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:10.262+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:10.272 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:10.267+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:10.533 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:10.528+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:10.588 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:10.583+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:10.673 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:10.668+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:10.774 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:10.770+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:10.821 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:10.816+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:11.241 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:11.236+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:11.305 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:11.299+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:11.513 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:11.508+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:11.569 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:11.564+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:11.655 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:11.650+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:11.767 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:11.762+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:11.855 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:11.850+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:12.229 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:12.224+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:12.291 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:12.286+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:12.536 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:12.530+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:12.604 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:12.597+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:12.678 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:12.673+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:12.813 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:12.808+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:12.816 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:12.811+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:13.268 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:13.263+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:13.279 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:13.274+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:13.562 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:13.552+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:13.598 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:13.593+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:13.673 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:13.666+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:13.799 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:13.794+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:13.864 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:13.859+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:14.256 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:14.251+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:14.269 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:14.264+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:14.559 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:14.555+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:14.586 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:14.582+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:14.648 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:14.643+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:14.759 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:14.755+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:14.837 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:14.830+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:15.234 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:15.229+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:15.249 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:15.240+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:15.522 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:15.517+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:15.637 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:15.632+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:15.676 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:15.671+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:15.756 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:15.750+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:15.800 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:15.795+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:16.231 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:16.226+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:16.241 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:16.233+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:16.574 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:16.569+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:16.677 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:16.672+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:16.688 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:16.683+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:16.710 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:16.705+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:16.832 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:16.827+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:17.227 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:17.222+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:17.252 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:17.247+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:17.596 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:17.592+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:17.643 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:17.639+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:17.688 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:17.684+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:17.760 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:17.752+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:17.837 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:17.832+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:18.222 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:18.217+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:18.246 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:18.241+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:18.597 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:18.592+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:18.620 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:18.615+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:18.727 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:18.721+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:18.798 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:18.792+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:18.803 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:18.798+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:19.183 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:19.179+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:19.207 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:19.203+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:19.585 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:19.580+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:19.589 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:19.584+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:19.738 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:19.732+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:19.806 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:19.800+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:19.853 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:19.847+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:20.170 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:20.165+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:20.211 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:20.200+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:20.544 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:20.538+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:20.630 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:20.625+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:20.767 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:20.760+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:20.805 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:20.801+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:20.861 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:20.856+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:21.142 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:21.136+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:21.182 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:21.177+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:21.522 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:21.516+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:21.684 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:21.678+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:21.771 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:21.766+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:21.838 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:21.833+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:21.846 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:21.841+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:22.102 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:22.096+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:22.223 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:22.216+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:22.563 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:22.557+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:22.715 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:22.711+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:22.745 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:22.737+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:22.807 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:22.802+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:22.885 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:22.880+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:23.106 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:23.100+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:23.180 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:23.174+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:23.543 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:23.532+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:23.711 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:23.707+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:23.724 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:23.719+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:23.859 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:23.854+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:23.883 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:23.878+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:24.105 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:24.099+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:24.174 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:24.167+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:24.501 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:24.495+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:24.712 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:24.708+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:24.729 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:24.717+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:24.884 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:24.878+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:24.913 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:24.908+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:25.030 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:32:24.935+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 44sec e355 v355) 2019-12-09T16:32:25.069 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:25.062+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:25.215 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:25.209+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:25.476 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:25.470+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:25.690 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:25.684+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:25.758 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:25.753+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:25.900 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:25.895+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:25.961 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:25.956+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:26.030 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:26.024+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:26.213 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:26.207+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:26.449 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:26.440+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:26.685 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:26.679+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:26.774 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:26.770+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:26.929 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:26.925+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:26.949 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:26.944+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:27.048 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:27.042+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:27.216 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:27.210+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:27.463 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:27.458+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:27.709 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:27.704+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:27.819 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:27.809+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:27.947 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:27.942+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:27.960 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:27.955+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:28.102 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:28.096+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:28.189 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:28.184+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:28.445 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:28.439+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:28.732 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:28.727+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:28.825 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:28.816+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:28.975 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:28.970+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:28.978 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:28.973+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:29.078 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:29.072+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:29.159 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:29.154+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:29.493 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:29.488+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:29.739 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:29.734+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:29.781 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:29.776+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:29.959 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:32:29.953+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 44sec e355 v355) 2019-12-09T16:32:30.007 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:30.000+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:30.019 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:30.015+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:30.056 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:30.051+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:30.184 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:30.178+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:30.542 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:30.536+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:30.770 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:30.763+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:30.783 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:30.778+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:31.014 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:31.009+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:31.038 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:31.034+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:31.093 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:31.088+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:31.227 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:31.222+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:31.533 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:31.527+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:31.820 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:31.814+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:31.827 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:31.820+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:31.983 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:31.979+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:32.062 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:32.057+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:32.120 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:32.111+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:32.207 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:32.200+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:32.501 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:32.489+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:32.799 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:32.794+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:32.866 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:32.860+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:32.943 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:32.938+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:33.072 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:33.065+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:33.136 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:33.130+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:33.171 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:33.166+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:33.533 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:33.528+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:33.821 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:33.816+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:33.844 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:33.839+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:33.956 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:33.950+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:34.043 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:34.038+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:34.119 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:34.112+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:34.171 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:34.165+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:34.528 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:34.521+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:34.788 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:34.783+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:34.829 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:34.820+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:34.946 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:34.941+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:35.038 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:32:35.032+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 44sec e355 v355) 2019-12-09T16:32:35.050 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:35.045+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:35.150 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:35.144+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:35.150 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:35.145+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:35.553 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:35.547+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:35.746 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:35.741+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:35.845 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:35.836+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:35.950 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:35.945+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:36.024 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:36.019+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:36.113 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:36.106+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:36.196 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:36.190+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:36.525 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:36.519+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:36.774 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:36.769+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:36.813 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:36.807+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:36.952 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:36.947+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:37.038 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:37.030+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:37.112 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:37.105+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:37.218 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:37.212+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:37.539 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:37.532+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:37.774 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:37.769+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:37.798 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:37.793+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:37.994 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:37.990+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:38.054 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:38.049+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:38.079 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:38.073+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:38.245 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:38.237+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:38.547 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:38.541+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:38.762 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:38.757+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:38.799 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:38.795+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:38.980 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:38.975+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:39.071 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:39.065+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:39.116 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:39.110+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:32:39.205 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:32:39.195+0000 1694f700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.156444+0000 front 2019-12-09T16:31:12.116520+0000 (oldest deadline 2019-12-09T16:31:52.523005+0000) 2019-12-09T16:32:39.591 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:32:39.586+0000 1694f700 -1 osd.4 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:08.817723+0000 front 2019-12-09T16:31:12.812842+0000 (oldest deadline 2019-12-09T16:31:54.283908+0000) 2019-12-09T16:32:39.774 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:32:39.768+0000 1694f700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.064622+0000 front 2019-12-09T16:31:07.062700+0000 (oldest deadline 2019-12-09T16:31:51.358108+0000) 2019-12-09T16:32:39.778 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:32:39.773+0000 1694f700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:07.662217+0000 front 2019-12-09T16:31:12.797951+0000 (oldest deadline 2019-12-09T16:31:50.870080+0000) 2019-12-09T16:32:40.022 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:32:40.016+0000 1694f700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.243734+0000 front 2019-12-09T16:31:11.221075+0000 (oldest deadline 2019-12-09T16:31:51.043233+0000) 2019-12-09T16:32:40.065 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:32:40.058+0000 1694f700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:10.238508+0000 front 2019-12-09T16:31:10.221988+0000 (oldest deadline 2019-12-09T16:31:50.676915+0000) 2019-12-09T16:32:40.122 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:32:40.114+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.22:6818/9815,v1:172.21.15.22:6819/9815] for 44sec e355 v355) 2019-12-09T16:32:40.132 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:32:40.127+0000 1694f700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.22:6823 osd.6 since back 2019-12-09T16:31:12.923510+0000 front 2019-12-09T16:31:06.727343+0000 (oldest deadline 2019-12-09T16:31:50.147891+0000) 2019-12-09T16:33:28.040 INFO:tasks.ceph.osd.6.smithi022.stderr:*** Caught signal (Aborted) ** 2019-12-09T16:33:28.040 INFO:tasks.ceph.osd.6.smithi022.stderr: in thread 3a280700 thread_name:tp_osd_tp 2019-12-09T16:33:31.020 INFO:tasks.ceph.osd.6.smithi022.stderr:daemon-helper: command crashed with signal 6 2019-12-09T16:33:33.105 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:33.101+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:34.134 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:34.129+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:35.155 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:35.150+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:35.506 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2019-12-09T16:33:36.168 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:36.164+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:37.208 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:37.204+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:38.197 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:38.193+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:39.237 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:39.233+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:40.234 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:40.230+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:41.232 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:41.228+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:41.831 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~6s 2019-12-09T16:33:42.194 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:42.190+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:43.239 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:43.234+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:44.191 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:44.187+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:45.171 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:45.166+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:46.135 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:46.131+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:47.116 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:47.112+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:48.135 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:48.130+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:48.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~13s 2019-12-09T16:33:49.119 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:49.113+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:50.150 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:50.143+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:51.189 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:51.181+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:52.226 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:52.221+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:53.226 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:53.220+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:54.248 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:54.243+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:54.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~19s 2019-12-09T16:33:55.293 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:55.283+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:56.326 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:56.322+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:57.318 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:57.314+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:58.364 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:58.359+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:33:59.378 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:33:59.373+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:00.371 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:00.367+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:00.770 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~25s 2019-12-09T16:34:01.341 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:01.337+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:02.346 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:02.342+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:03.361 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:03.356+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:04.380 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:04.375+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:05.351 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:05.347+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:06.398 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:06.394+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:07.086 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~32s 2019-12-09T16:34:07.362 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:07.357+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:08.321 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:08.317+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:09.287 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:09.281+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:10.262 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:10.257+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:11.260 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:11.255+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:12.227 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:12.220+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:13.195 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:13.190+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:13.401 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~38s 2019-12-09T16:34:14.194 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:14.190+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:15.193 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:15.189+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:16.199 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:16.194+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:17.205 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:17.199+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:18.184 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:18.178+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:19.150 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:19.143+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:19.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~44s 2019-12-09T16:34:20.192 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:20.188+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:21.178 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:21.174+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:22.138 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:22.132+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:23.130 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:23.125+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:23.795 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:23.789+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:24.142 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:24.136+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:24.144 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:24.139+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:24.643 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:24.635+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:24.659 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:24.651+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:24.828 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:24.822+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:25.102 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:25.096+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:25.102 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:25.098+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:25.661 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:25.654+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:25.708 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:25.701+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:25.827 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:25.822+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:25.915 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:25.907+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:26.023 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~51s 2019-12-09T16:34:26.093 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:26.085+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:26.093 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:26.089+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:26.666 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:26.659+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:26.758 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:26.752+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:26.783 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:26.778+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:26.944 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:26.938+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:27.111 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:27.105+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:27.112 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:27.108+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:27.541 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:27.536+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:27.677 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:27.670+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:27.746 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:27.738+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:27.753 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:27.748+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:27.929 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:27.923+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:28.079 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:28.073+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:28.084 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:28.079+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:28.555 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:28.548+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:28.653 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:28.646+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:28.772 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:28.765+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:28.793 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:28.787+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:28.884 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:28.878+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:29.068 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:29.063+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:29.073 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:29.068+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:29.522 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:29.517+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:29.630 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:29.623+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:29.767 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:29.760+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:29.783 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:29.778+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:29.936 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:29.929+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:30.048 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:30.043+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:30.048 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:30.044+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:30.548 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:30.537+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:30.615 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:30.606+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:30.790 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:30.783+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:30.806 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:30.800+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:30.970 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:30.963+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:31.082 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:31.072+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:31.084 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:31.078+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:31.569 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:31.563+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:31.612 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:31.605+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:31.752 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:31.747+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:31.786 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:31.779+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:31.989 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:31.982+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:32.051 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:32.043+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:32.052 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:32.047+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:32.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~57s 2019-12-09T16:34:32.592 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:32.586+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:32.644 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:32.637+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:32.766 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:32.759+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:32.801 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:32.794+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:32.985 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:32.978+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:33.096 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:33.092+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:33.101 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:33.096+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:33.601 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:33.595+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:33.626 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:33.620+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:33.781 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:33.775+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:33.816 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:33.809+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:33.943 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:33.937+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:34.079 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:34.072+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:34.080 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:34.075+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:34.615 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:34.609+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:34.657 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:34.639+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:34.815 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:34.809+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:34.825 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:34.818+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:34.942 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:34.932+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:35.075 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:35.070+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:35.076 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:35.072+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:35.579 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:35.572+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:35.608 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:35.603+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:35.828 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:35.822+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:35.836 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:35.831+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:35.973 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:35.952+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:36.107 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:36.099+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:36.109 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:36.104+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:36.575 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:36.569+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:36.656 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:36.650+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:36.823 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:36.815+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:36.826 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:36.820+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:36.992 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:36.986+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:37.064 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:37.056+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:37.066 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:37.060+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:37.562 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:37.556+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:37.627 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:37.621+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:37.800 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:37.795+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:37.829 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:37.823+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:38.000 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:37.995+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:38.046 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:38.039+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:38.048 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:38.043+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:38.576 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:38.571+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:38.592 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:38.586+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:38.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~63s 2019-12-09T16:34:38.801 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:38.795+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:38.811 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:38.805+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:39.006 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:38.999+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:39.008 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:39.003+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:39.036 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:39.031+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:39.555 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:39.547+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:39.628 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:39.622+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:39.771 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:39.766+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:39.779 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:39.773+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:40.002 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:39.997+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:40.004 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:39.999+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:40.009 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:40.003+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:40.554 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:40.549+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:40.593 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:40.588+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:40.770 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:40.765+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:40.821 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:40.816+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:40.980 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:40.972+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:41.045 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:41.041+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:41.047 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:41.044+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:41.554 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:41.546+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:41.577 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:41.567+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:41.753 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:41.748+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:41.866 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:41.859+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:41.998 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:41.992+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:42.029 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:42.023+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:42.032 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:42.025+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:42.513 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:42.508+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:42.540 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:42.534+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:42.725 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:42.719+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:42.912 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:42.904+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:43.001 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:42.996+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:43.067 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:43.059+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:43.069 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:43.063+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:43.531 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:43.524+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:43.564 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:43.559+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:43.679 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:43.674+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:43.942 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:43.937+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:44.046 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:44.039+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:44.078 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:44.072+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:44.080 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:44.075+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:44.575 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:44.569+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:44.617 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:44.609+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:44.726 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:44.720+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:44.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~69s 2019-12-09T16:34:44.988 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:44.982+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:45.084 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:45.076+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:45.111 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:45.103+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:45.111 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:45.106+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:45.532 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:45.526+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:45.581 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:45.575+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:45.710 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:45.704+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:45.968 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:45.962+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:46.075 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:46.068+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:46.076 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:46.071+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 14 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:46.113 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:46.107+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:46.571 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:46.565+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:46.578 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:46.572+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:46.740 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:46.735+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:46.937 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:46.931+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:47.069 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:47.061+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:47.076 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:47.065+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:47.089 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:47.084+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:47.558 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:47.553+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:47.571 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:47.562+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:47.778 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:47.772+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:47.966 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:47.961+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:48.076 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:48.069+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:48.077 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:48.073+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:48.095 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:48.089+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:48.529 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:48.523+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:48.597 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:48.592+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:48.793 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:48.787+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:48.944 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:48.937+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:49.099 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:49.092+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:49.101 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:49.096+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:49.102 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:49.097+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:49.539 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:49.533+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:49.557 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:49.551+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:49.835 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:49.830+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:49.951 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:34:49.946+0000 1694f700 -1 osd.4 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:39.575159+0000 front 2019-12-09T16:33:39.661813+0000 (oldest deadline 2019-12-09T16:34:24.099241+0000) 2019-12-09T16:34:50.111 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:50.106+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.658095+0000 front 2019-12-09T16:33:41.662797+0000 (oldest deadline 2019-12-09T16:34:23.957203+0000) 2019-12-09T16:34:50.117 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:50.112+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:50.141 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:50.134+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:50.532 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:34:50.525+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:41.396717+0000 front 2019-12-09T16:33:41.655381+0000 (oldest deadline 2019-12-09T16:34:27.111838+0000) 2019-12-09T16:34:50.597 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:34:50.587+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.469282+0000 front 2019-12-09T16:33:40.415734+0000 (oldest deadline 2019-12-09T16:34:24.246945+0000) 2019-12-09T16:34:50.895 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:34:50.889+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.746105+0000 front 2019-12-09T16:33:40.763014+0000 (oldest deadline 2019-12-09T16:34:23.629329+0000) 2019-12-09T16:34:51.109 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:34:51.103+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:33:40.269589+0000 front 2019-12-09T16:33:40.153637+0000 (oldest deadline 2019-12-09T16:34:25.308250+0000) 2019-12-09T16:34:51.276 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~76s 2019-12-09T16:34:51.510 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:51.354+0000 1694f700 -1 osd.0 358 get_health_metrics reporting 15 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:52.479 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:34:52.415+0000 1694f700 -1 osd.0 359 get_health_metrics reporting 16 slow ops, oldest is osd_op(client.4355.0:119227 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5458454224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e357) 2019-12-09T16:34:53.474 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:34:53.460+0000 20e25700 -1 RGWWatcher::handle_error cookie 5458454224 err (107) Transport endpoint is not connected 2019-12-09T16:34:57.590 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~82s 2019-12-09T16:35:03.901 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~88s 2019-12-09T16:35:05.270 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:35:05.262+0000 34274700 -1 osd.1 362 build_incremental_map_msg missing incremental map 362 2019-12-09T16:35:05.424 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:35:05.385+0000 3a280700 -1 osd.1 362 build_incremental_map_msg missing incremental map 362 2019-12-09T16:35:05.458 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:35:05.402+0000 35276700 -1 osd.1 362 build_incremental_map_msg missing incremental map 362 2019-12-09T16:35:05.500 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:35:05.445+0000 3aa81700 -1 osd.1 362 build_incremental_map_msg missing incremental map 362 2019-12-09T16:35:05.532 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:35:05.518+0000 35276700 -1 osd.1 362 build_incremental_map_msg missing incremental map 362 2019-12-09T16:35:05.637 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:35:05.587+0000 35a77700 -1 osd.1 362 build_incremental_map_msg missing incremental map 362 2019-12-09T16:35:06.618 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:35:06.606+0000 36278700 -1 osd.1 362 build_incremental_map_msg missing incremental map 362 2019-12-09T16:35:09.572 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:35:09.543+0000 36278700 -1 osd.1 362 build_incremental_map_msg missing incremental map 362 2019-12-09T16:35:10.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~95s 2019-12-09T16:35:16.527 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~101s 2019-12-09T16:35:22.844 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~107s 2019-12-09T16:35:29.155 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~114s 2019-12-09T16:35:35.467 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~120s 2019-12-09T16:35:36.119 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:35:35.853+0000 1694f700 -1 osd.1 362 get_health_metrics reporting 24 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=57 ondisk+retry+read+known_if_redirected e361) 2019-12-09T16:35:41.779 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~126s 2019-12-09T16:35:48.092 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~133s 2019-12-09T16:35:54.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~139s 2019-12-09T16:36:00.721 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~145s 2019-12-09T16:36:07.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~152s 2019-12-09T16:36:13.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~158s 2019-12-09T16:36:19.659 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~164s 2019-12-09T16:36:25.234 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:25.230+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:25.969 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~170s 2019-12-09T16:36:26.265 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:26.260+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:26.428 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:24:02.692 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:26.435 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:24:02.699 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:26.455 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:24:02.677 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:26.457 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:24:02.679 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:26.465 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:24:02.687 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:26.465 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:24:02.687 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:26.785 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:24:03.049 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:26.838 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:24:03.060 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:26.844 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:24:03.066 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:27.299 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:27.296+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:27.479 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:27.471+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:27.945 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:27.940+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:28.261 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:28.257+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:28.495 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:28.488+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:28.972 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:28.966+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:29.226 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:29.223+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:29.539 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:29.531+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:29.902 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:29.897+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:30.021 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:30.016+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:30.136 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:30.130+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:30.197 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:30.192+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:30.564 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:30.558+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:30.866 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:30.860+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:31.032 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:31.027+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:31.121 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:31.114+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:31.181 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:31.176+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:31.580 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:31.574+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:31.902 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:31.896+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:32.017 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:32.012+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:32.155 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:32.149+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:32.157 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:32.152+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:32.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~177s 2019-12-09T16:36:32.602 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:32.595+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:32.947 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:32.940+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:32.994 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:32.988+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:33.169 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:33.165+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:33.179 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:33.173+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:33.607 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:33.599+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:33.906 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:33.902+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:34.035 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:34.030+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:34.180 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:34.176+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:34.192 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:34.186+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:34.638 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:34.631+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:34.958 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:34.954+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:35.036 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:35.030+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:35.152 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:35.148+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:35.190 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:35.183+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:35.610 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:35.603+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:35.978 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:35.969+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:36.074 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:36.070+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:36.119 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:36.114+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:36.152 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:36.145+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:36.585 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:36.578+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:37.031 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:37.024+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:37.041 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:37.037+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:37.171 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:37.167+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:37.192 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:37.184+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:37.628 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:37.622+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:38.066 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:38.059+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:38.070 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:38.065+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:38.185 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:38.178+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:38.215 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:38.208+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:38.219 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:38.215+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:38.597 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~183s 2019-12-09T16:36:38.616 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:38.605+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:39.077 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:39.071+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:39.087 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:39.081+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:39.171 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:39.163+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:39.252 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:39.247+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:39.255 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:39.251+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:39.575 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:39.567+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:40.088 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:40.083+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:40.110 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:40.105+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:40.209 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:40.202+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:40.279 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:40.272+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:40.280 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:40.275+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:40.576 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:40.569+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:41.094 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:41.089+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:41.129 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:41.124+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:41.168 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:41.162+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:41.249 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:41.242+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:41.250 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:41.245+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:41.616 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:41.599+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:42.062 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:42.056+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:42.099 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:42.093+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:42.180 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:42.172+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:42.265 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:42.257+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:42.266 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:42.261+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:42.570 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:42.562+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:43.063 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:43.057+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:43.127 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:43.121+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:43.165 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:43.158+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:43.287 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:43.282+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:43.290 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:43.285+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:43.612 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:43.606+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:44.077 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:44.071+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:44.130 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:44.124+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:44.212 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:44.205+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:44.279 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:44.270+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:44.280 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:44.274+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:44.623 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:44.615+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:44.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~189s 2019-12-09T16:36:45.036 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:45.029+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:45.148 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:45.142+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:45.193 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:45.186+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:45.320 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:45.313+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:45.321 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:45.316+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:45.633 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:45.626+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:46.072 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:46.067+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:46.124 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:46.118+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:46.152 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:46.145+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:46.375 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:46.367+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:46.380 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:46.371+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:46.635 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:46.629+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:47.019 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:36:47.011+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:43.004933+0000 front 2019-12-09T16:35:42.990677+0000 (oldest deadline 2019-12-09T16:36:46.294731+0000) 2019-12-09T16:36:47.031 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:47.026+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:47.105 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:47.099+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:47.186 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:47.180+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:47.344 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:47.337+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:47.344 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:47.340+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:47.614 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:47.607+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:48.007 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:48.002+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:48.016 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:36:48.009+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:43.004933+0000 front 2019-12-09T16:35:42.990677+0000 (oldest deadline 2019-12-09T16:36:46.294731+0000) 2019-12-09T16:36:48.122 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:48.115+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:48.144 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:48.137+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:48.336 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:48.330+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:48.340 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:48.336+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:48.650 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:48.643+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:48.987 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:48.982+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:49.057 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:36:49.051+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:43.004933+0000 front 2019-12-09T16:35:42.990677+0000 (oldest deadline 2019-12-09T16:36:46.294731+0000) 2019-12-09T16:36:49.153 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:49.148+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:49.193 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:49.185+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:49.386 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:49.379+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:49.387 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:49.383+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:49.700 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:49.692+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:49.977 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:49.971+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:50.049 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:36:50.042+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:43.004933+0000 front 2019-12-09T16:35:42.990677+0000 (oldest deadline 2019-12-09T16:36:46.294731+0000) 2019-12-09T16:36:50.148 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:50.142+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:50.158 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:50.153+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:50.411 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:50.405+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:50.412 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:50.407+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:50.657 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:50.650+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:50.985 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:36:50.980+0000 1694f700 -1 osd.3 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.342626+0000 front 2019-12-09T16:35:41.405392+0000 (oldest deadline 2019-12-09T16:36:27.105673+0000) 2019-12-09T16:36:51.014 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:36:51.008+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:43.004933+0000 front 2019-12-09T16:35:42.990677+0000 (oldest deadline 2019-12-09T16:36:46.294731+0000) 2019-12-09T16:36:51.120 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:36:51.113+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.413362+0000 front 2019-12-09T16:35:44.415284+0000 (oldest deadline 2019-12-09T16:36:30.074662+0000) 2019-12-09T16:36:51.155 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:36:51.149+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:46.906878+0000 front 2019-12-09T16:35:46.109865+0000 (oldest deadline 2019-12-09T16:36:29.561659+0000) 2019-12-09T16:36:51.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~196s 2019-12-09T16:36:51.403 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:51.395+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:44.412772+0000 front 2019-12-09T16:35:44.465212+0000 (oldest deadline 2019-12-09T16:36:37.293353+0000) 2019-12-09T16:36:51.404 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:36:51.399+0000 1694f700 -1 osd.0 362 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:119497 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5439379488 gen 1] snapc 0=[] ondisk+write+known_if_redirected e362) 2019-12-09T16:36:51.627 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:36:51.620+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:35:42.684269+0000 front 2019-12-09T16:35:42.652297+0000 (oldest deadline 2019-12-09T16:36:26.692864+0000) 2019-12-09T16:36:53.959 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:36:53.955+0000 20e25700 -1 RGWWatcher::handle_error cookie 5439379488 err (107) Transport endpoint is not connected 2019-12-09T16:36:57.533 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~202s 2019-12-09T16:37:03.845 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~208s 2019-12-09T16:37:06.739 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:24:43.003 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:37:06.749 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:24:43.013 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:37:06.778 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:24:43.000 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:37:06.795 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:24:43.017 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:37:06.798 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:24:43.020 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:37:06.809 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:24:43.031 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:37:06.864 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:24:43.086 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:37:06.913 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:24:43.135 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:37:06.976 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:24:43.241 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:37:10.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~215s 2019-12-09T16:37:16.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~221s 2019-12-09T16:37:22.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~227s 2019-12-09T16:37:29.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~234s 2019-12-09T16:37:35.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~240s 2019-12-09T16:37:41.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~246s 2019-12-09T16:37:48.043 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~253s 2019-12-09T16:37:54.355 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~259s 2019-12-09T16:38:00.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~265s 2019-12-09T16:38:06.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~271s 2019-12-09T16:38:13.300 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~278s 2019-12-09T16:38:19.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~284s 2019-12-09T16:38:25.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~290s 2019-12-09T16:38:32.231 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~297s 2019-12-09T16:38:38.543 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~303s 2019-12-09T16:38:38.543 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2019-12-09T16:38:38.544 ERROR:tasks.daemonwatchdog.daemon_watchdog:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 38, in _run self.watch() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 114, in watch self.bark() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 54, in bark for mount in self.ctx.mounts.values(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/config.py", line 253, in __getattr__ raise AttributeError(name) AttributeError: mounts 2019-12-09T16:39:17.138 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:17.131+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:17.468 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:17.463+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:17.518 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:17.513+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:18.137 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:18.131+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:18.481 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:18.477+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:18.492 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:18.487+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:18.492 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:18.489+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:18.586 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:18.580+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:18.596 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:18.589+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:19.159 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:19.153+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:19.480 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:19.473+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:19.486 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:19.481+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:19.489 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:19.485+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:19.600 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:19.594+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:19.628 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:19.622+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:20.200 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:20.193+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:20.445 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:20.438+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:20.488 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:20.481+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:20.489 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:20.485+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:20.617 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:20.610+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:20.645 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:20.637+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:21.164 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:21.156+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:21.243 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:21.236+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:21.436 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:21.431+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:21.447 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:21.443+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:21.449 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:21.445+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:21.602 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:21.596+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:21.618 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:21.610+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:22.181 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:22.174+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:22.296 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:22.289+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:22.488 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:22.482+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:22.491 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:22.484+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:22.493 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:22.488+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:22.568 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:22.563+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:22.647 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:22.640+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:23.217 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:23.210+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:23.267 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:23.261+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:23.458 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:23.452+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:23.458 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:23.454+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:23.470 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:23.465+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:23.534 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:23.528+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:23.636 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:23.630+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:24.219 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:24.211+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:24.285 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:24.279+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:24.452 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:24.447+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:24.474 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:24.468+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:24.475 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:24.471+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:24.495 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:24.490+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:24.688 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:24.682+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:25.258 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:25.251+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:25.273 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:25.267+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:25.434 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:25.428+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:25.434 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:25.430+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:25.476 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:25.470+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:25.524 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:25.518+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:25.765 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:25.752+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:26.288 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:26.283+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:26.294 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:26.287+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:26.433 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:26.427+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:26.433 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:26.429+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:26.463 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:26.458+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:26.483 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:26.479+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:26.799 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:26.792+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:27.274 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:27.266+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:27.277 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:27.271+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:27.438 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:27.434+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:27.460 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:27.454+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:27.461 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:27.457+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:27.491 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:27.485+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:27.792 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:27.785+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:28.255 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:28.247+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:28.271 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:28.264+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:28.422 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:28.417+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:28.467 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:28.462+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:28.467 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:28.464+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:28.537 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:28.533+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:28.764 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:28.757+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:29.289 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:29.282+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:29.315 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:29.307+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:29.401 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:29.396+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:29.463 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:29.457+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:29.464 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:29.459+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:29.512 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:29.506+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:29.790 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:29.783+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:30.288 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:30.281+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:30.330 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:30.324+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:30.441 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:30.436+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:30.452 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:30.446+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:30.453 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:30.449+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:30.515 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:30.511+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:30.757 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:30.751+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:31.296 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:31.289+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:31.348 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:31.341+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:31.439 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:31.435+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:31.441 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:31.438+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:31.490 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:31.485+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:31.547 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:31.542+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:31.767 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:31.760+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:32.293 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:32.286+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:32.347 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:32.341+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:32.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:32.390+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:32.399 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:32.396+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:32.506 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:32.501+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:32.556 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:32.550+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:32.813 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:32.806+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:33.291 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:33.285+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:33.310 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:33.303+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:33.368 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:33.361+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:33.369 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:33.364+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:33.549 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:33.543+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:33.561 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:33.554+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:33.769 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:33.763+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:34.335 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:34.326+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:34.355 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:34.340+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:34.384 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:34.378+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:34.385 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:34.380+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:34.552 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:34.546+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:34.553 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:34.548+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:34.798 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:34.791+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:35.355 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:35.348+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:35.363 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:35.358+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:35.366 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:35.362+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:35.408 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:35.400+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:35.592 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:35.587+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:35.599 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:35.594+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:35.828 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:35.822+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:36.355 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:36.348+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:36.356 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:36.351+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:36.381 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:36.374+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:36.402 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:36.395+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:36.596 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:36.590+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:36.643 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:36.638+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:36.872 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:36.865+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:37.331 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:37.325+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:37.334 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:37.330+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:37.382 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:37.374+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:37.426 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:37.421+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:37.612 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:37.606+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:37.674 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:37.668+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:37.849 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:37.842+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:38.300 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:38.294+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:38.301 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:38.296+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:38.415 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:38.409+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:38.442 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:38.436+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:38.652 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:38.647+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:38.728 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:38.723+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:38.858 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:38.850+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:39.282 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:39.276+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:39.286 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:39.281+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:39.438 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:39.431+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:39.495 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:39.488+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:39.614 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:39.609+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:39.771 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:39.766+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:39.912 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:39.905+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:40.245 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:40.238+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:40.245 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:40.241+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:40.409 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:40.402+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:40.481 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:40.471+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:40.646 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:40.641+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:40.810 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:40.804+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:40.916 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:40.909+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:41.218 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:41.212+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:41.218 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:41.214+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:41.433 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:41.425+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:41.474 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:41.461+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:41.697 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:41.693+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:41.790 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:41.785+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:41.910 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:41.904+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:42.181 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:42.174+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:42.182 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:42.177+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:42.457 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:42.450+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:42.459 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:42.451+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:42.662 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:42.657+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:42.756 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:42.751+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:42.925 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:42.918+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:43.233 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:43.228+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:43.236 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:43.232+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:43.441 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:43.434+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:43.505 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:43.498+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:43.635 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:43.630+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:43.724 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:43.717+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:43.935 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:43.926+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:44.242 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:44.238+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:44.244 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:44.241+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:44.463 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:44.456+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:44.490 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:44.484+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:44.604 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:44.599+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:44.689 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:44.681+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:44.920 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:44.913+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:45.215 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:45.208+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:45.216 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:45.212+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:45.501 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:45.494+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:45.504 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:45.497+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:45.630 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:45.625+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:45.689 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:45.683+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:45.948 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:45.942+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:46.176 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:46.170+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:46.177 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:46.173+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:46.458 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:46.452+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:46.465 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:46.460+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:46.624 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:46.619+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:46.649 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:46.644+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:46.928 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:46.921+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:47.150 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:47.143+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:47.151 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:47.145+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:47.462 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:47.457+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:47.498 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:47.492+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:47.647 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:47.642+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:47.686 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:47.681+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:47.913 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:47.909+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:48.110 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:48.105+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:48.112 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:48.107+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:48.422 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:48.416+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:48.493 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:48.487+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:48.609 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:48.603+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:48.725 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:48.719+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:48.938 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:48.933+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:49.113 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:49.107+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:49.117 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:49.112+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:49.441 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:49.434+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:49.463 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:49.458+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:49.628 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:49.622+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:49.712 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:49.708+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:49.933 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:49.927+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:50.116 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:50.110+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:50.120 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:50.116+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:50.462 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:50.456+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:50.487 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:50.481+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:50.673 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:50.668+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:50.761 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:50.755+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:50.894 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:50.889+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:51.083 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:51.077+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:51.084 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:51.079+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:51.458 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:51.454+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:51.483 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:51.476+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:51.646 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:51.641+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:51.749 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:39:51.745+0000 1694f700 -1 osd.3 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:32.114585+0000 front 2019-12-09T16:38:36.932018+0000 (oldest deadline 2019-12-09T16:39:17.363019+0000) 2019-12-09T16:39:51.856 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:51.850+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:52.083 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:52.076+0000 1694f700 -1 osd.0 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.965839+0000 front 2019-12-09T16:38:34.815767+0000 (oldest deadline 2019-12-09T16:39:18.085858+0000) 2019-12-09T16:39:52.085 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:52.080+0000 1694f700 -1 osd.0 367 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:52.474 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:39:52.468+0000 1694f700 -1 osd.5 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.934519+0000 front 2019-12-09T16:38:36.932930+0000 (oldest deadline 2019-12-09T16:39:20.403755+0000) 2019-12-09T16:39:52.475 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:39:52.470+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 47sec e367 v367) 2019-12-09T16:39:52.500 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:39:52.495+0000 1694f700 -1 osd.7 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:36.931066+0000 front 2019-12-09T16:38:36.542122+0000 (oldest deadline 2019-12-09T16:39:16.428509+0000) 2019-12-09T16:39:52.683 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:39:52.678+0000 1694f700 -1 osd.2 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:33.714174+0000 front 2019-12-09T16:38:37.708002+0000 (oldest deadline 2019-12-09T16:39:18.402024+0000) 2019-12-09T16:39:52.853 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:39:52.849+0000 1694f700 -1 osd.4 367 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:38:34.830961+0000 front 2019-12-09T16:38:34.824884+0000 (oldest deadline 2019-12-09T16:39:17.695340+0000) 2019-12-09T16:39:53.399 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:53.394+0000 1694f700 -1 osd.0 368 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:54.544 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:39:54.523+0000 1694f700 -1 osd.0 369 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:119773 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441778768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e367) 2019-12-09T16:39:54.895 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:39:54.874+0000 20e25700 -1 RGWWatcher::handle_error cookie 5441778768 err (107) Transport endpoint is not connected 2019-12-09T16:40:41.061 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:28:17.325 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:40:41.076 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:28:17.298 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:40:41.077 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:28:17.342 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:40:41.080 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:28:17.303 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:40:41.097 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:28:17.319 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:40:41.111 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:28:17.333 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:40:41.296 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:28:17.518 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:40:41.351 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:28:17.573 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:40:41.376 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:28:17.641 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:42:09.273 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:09.266+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:09.664 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:09.654+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:10.272 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:10.265+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:10.682 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:10.676+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:10.923 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:10.918+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:11.282 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:11.276+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:11.668 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:11.661+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:11.882 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:11.877+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:12.036 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:12.029+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:12.257 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:12.250+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:12.258 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:12.253+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:12.715 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:12.708+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:12.878 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:12.873+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:12.959 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:12.955+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:13.055 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:13.048+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:13.112 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:42:13.105+0000 1694f700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.365798+0000 front 2019-12-09T16:41:27.833047+0000 (oldest deadline 2019-12-09T16:42:12.815651+0000) 2019-12-09T16:42:13.249 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:13.244+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:13.252 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:13.247+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:13.725 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:13.717+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:13.909 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:13.903+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:13.971 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:13.966+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:14.082 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:14.075+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:14.150 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:42:14.143+0000 1694f700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.365798+0000 front 2019-12-09T16:41:27.833047+0000 (oldest deadline 2019-12-09T16:42:12.815651+0000) 2019-12-09T16:42:14.221 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:14.215+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:14.222 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:14.217+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:14.752 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:14.746+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:14.932 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:14.925+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:15.010 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:15.004+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:15.058 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:15.051+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:15.178 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:42:15.170+0000 1694f700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.365798+0000 front 2019-12-09T16:41:27.833047+0000 (oldest deadline 2019-12-09T16:42:12.815651+0000) 2019-12-09T16:42:15.230 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:15.225+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:15.231 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:15.227+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:15.794 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:15.788+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:15.890 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:15.886+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:16.026 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:16.020+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:16.054 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:16.047+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:16.183 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:42:16.176+0000 1694f700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.365798+0000 front 2019-12-09T16:41:27.833047+0000 (oldest deadline 2019-12-09T16:42:12.815651+0000) 2019-12-09T16:42:16.286 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:16.278+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:16.291 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:16.286+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:16.782 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:16.775+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:16.916 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:16.911+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:17.033 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:17.026+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:17.055 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:17.049+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:17.185 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:42:17.179+0000 1694f700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.365798+0000 front 2019-12-09T16:41:27.833047+0000 (oldest deadline 2019-12-09T16:42:12.815651+0000) 2019-12-09T16:42:17.283 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:17.276+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:17.284 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:17.279+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:17.823 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:17.815+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:17.884 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:17.878+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:18.072 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:18.065+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:18.091 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:18.085+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:18.241 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:42:18.234+0000 1694f700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.365798+0000 front 2019-12-09T16:41:27.833047+0000 (oldest deadline 2019-12-09T16:42:12.815651+0000) 2019-12-09T16:42:18.252 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:18.245+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:18.253 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:18.248+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:18.842 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:18.836+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:18.922 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:18.916+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:19.031 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:19.024+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:19.133 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:19.123+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:19.228 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:19.221+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:19.228 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:19.224+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:19.850 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:19.843+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:19.936 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:19.931+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:19.988 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:19.982+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:20.139 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:20.133+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:20.237 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:20.231+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:20.240 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:20.236+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:20.852 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:20.846+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:20.921 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:20.917+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:21.000 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:20.992+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:21.119 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:21.114+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:21.272 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:21.265+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:21.275 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:21.270+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:21.817 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:21.811+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:21.891 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:21.885+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:22.005 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:22.000+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:22.171 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:22.166+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:22.292 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:22.284+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:22.293 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:22.287+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:22.810 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:22.804+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:22.854 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:22.848+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:23.007 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:23.001+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:23.154 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:23.147+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:23.252 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:23.246+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:23.255 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:23.250+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:23.851 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:23.847+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:23.890 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:23.883+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:24.040 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:24.035+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:24.118 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:24.111+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:24.256 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:24.250+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:24.257 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:24.252+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:24.903 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:24.898+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:24.904 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:24.900+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:25.096 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:25.086+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:25.107 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:25.101+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:25.243 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:25.236+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:25.244 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:25.239+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:25.910 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:25.904+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:25.916 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:25.911+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:26.098 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:26.092+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:26.144 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:26.139+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:26.262 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:26.256+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:26.263 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:26.258+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:26.890 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:26.885+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:26.957 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:26.952+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:27.085 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:27.081+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:27.190 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:27.185+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:27.250 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:27.243+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:27.251 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:27.246+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:27.924 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:27.920+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:27.925 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:27.920+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:28.088 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:28.082+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:28.164 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:28.159+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:28.242 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:28.235+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:28.244 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:28.239+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:28.894 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:28.889+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:28.933 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:28.929+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:29.048 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:29.043+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:29.218 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:29.212+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:29.295 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:29.288+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:29.295 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:29.291+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:29.935 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:29.929+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:29.979 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:29.974+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:30.005 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:30.001+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:30.276 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:30.267+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:30.344 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:30.339+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:30.345 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:30.341+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:30.975 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:30.970+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:31.010 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:31.005+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:31.030 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:31.024+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:31.273 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:31.267+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:31.325 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:31.319+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:31.326 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:31.321+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:31.955 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:31.950+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:31.998 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:31.993+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:32.064 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:32.059+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:32.252 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:32.247+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:32.310 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:32.305+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:32.311 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:32.307+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:32.961 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:32.951+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:32.988 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:32.982+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:33.106 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:33.101+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:33.227 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:33.221+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:33.332 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:33.326+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:33.333 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:33.329+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:33.948 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:33.943+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:34.030 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:34.024+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:34.128 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:34.123+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:34.282 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:34.277+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:34.332 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:34.326+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:34.333 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:34.327+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:34.916 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:34.911+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:34.993 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:34.987+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:35.122 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:35.116+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:35.284 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:35.279+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:35.352 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:35.346+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:35.353 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:35.349+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:35.881 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:35.875+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:35.985 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:35.979+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:36.083 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:36.077+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:36.323 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:36.316+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:36.324 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:36.319+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:36.325 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:36.320+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:36.899 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:36.894+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:37.014 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:37.009+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:37.072 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:37.065+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:37.304 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:37.297+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:37.304 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:37.300+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:37.377 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:37.370+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:37.899 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:37.894+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:37.990 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:37.985+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:38.027 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:38.020+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:38.301 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:38.295+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:38.302 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:38.298+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:38.374 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:38.367+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:38.857 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:38.852+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:39.019 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:39.014+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:39.029 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:39.022+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:39.337 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:39.330+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:39.340 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:39.335+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:39.343 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:39.340+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:39.868 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:39.862+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:39.977 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:39.973+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:40.035 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:40.026+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:40.307 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:40.300+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:40.309 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:40.302+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:40.310 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:40.305+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:40.838 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:40.832+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:40.988 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:40.983+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:41.019 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:41.012+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:41.309 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:41.297+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:41.329 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:41.324+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:41.333 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:41.329+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:41.833 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:41.828+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:41.991 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:41.987+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:42.062 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:42.057+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:42.286 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:42.279+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:42.303 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:42.298+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:42.305 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:42.301+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:42.886 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:42.879+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:42.975 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:42.968+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:43.068 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:43.061+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:43.286 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:43.279+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:43.287 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:43.282+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:43.318 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:43.311+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:43.561 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:42:43.552+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 37sec e372 v372) 2019-12-09T16:42:43.851 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:43.845+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:44.004 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:43.999+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:44.096 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:44.089+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:44.247 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:44.243+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:44.250 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:44.247+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:44.342 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:44.336+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:44.808 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:44.804+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:44.988 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:44.983+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:45.052 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:45.045+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:45.232 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:45.227+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:45.235 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:45.231+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:45.336 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:45.330+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:45.820 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:45.815+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:45.967 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:45.962+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:46.067 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:46.059+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:46.197 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:46.190+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:46.198 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:46.194+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:46.299 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:46.293+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:46.858 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:46.853+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:46.947 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:46.941+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:47.107 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:47.101+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:47.178 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:47.172+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:47.179 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:47.175+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:47.338 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:47.331+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:47.864 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:47.859+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:47.902 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:47.896+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:48.148 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:48.140+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:48.178 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:48.171+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:48.178 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:48.174+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:48.339 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:48.332+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:48.580 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:42:48.576+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 37sec e372 v372) 2019-12-09T16:42:48.822 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:48.816+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:48.932 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:48.927+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:49.199 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:49.192+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:49.200 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:49.189+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:49.206 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:49.193+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:49.334 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:49.327+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:49.793 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:49.787+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:49.890 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:49.885+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:50.226 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:50.210+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:50.227 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:50.213+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:50.253 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:50.245+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:50.370 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:50.362+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:50.751 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:50.745+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:50.848 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:50.843+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:51.216 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:51.208+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:51.220 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:51.212+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:51.230 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:51.223+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:51.416 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:51.409+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:51.721 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:51.716+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:51.806 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:51.800+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:52.227 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:52.221+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:52.228 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:52.223+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:52.252 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:52.246+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:52.421 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:52.414+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:52.757 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:52.751+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:52.826 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:52.821+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:53.259 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:53.252+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:53.263 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:53.257+0000 1694f700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.340409+0000 front 2019-12-09T16:41:31.359434+0000 (oldest deadline 2019-12-09T16:42:08.425285+0000) 2019-12-09T16:42:53.264 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:53.260+0000 1694f700 -1 osd.0 372 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:53.431 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:53.424+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:53.648 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:42:53.642+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 37sec e372 v372) 2019-12-09T16:42:53.778 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:42:53.772+0000 1694f700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:32.366522+0000 front 2019-12-09T16:41:26.639863+0000 (oldest deadline 2019-12-09T16:42:12.385972+0000) 2019-12-09T16:42:53.785 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:42:53.779+0000 1694f700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:33.254377+0000 front 2019-12-09T16:41:32.656063+0000 (oldest deadline 2019-12-09T16:42:10.327296+0000) 2019-12-09T16:42:54.243 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:42:54.238+0000 1694f700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.999438+0000 front 2019-12-09T16:41:27.019698+0000 (oldest deadline 2019-12-09T16:42:08.962822+0000) 2019-12-09T16:42:54.397 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:54.346+0000 1694f700 -1 osd.0 373 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:54.435 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:42:54.429+0000 1694f700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:41:28.772232+0000 front 2019-12-09T16:41:32.366015+0000 (oldest deadline 2019-12-09T16:42:11.796768+0000) 2019-12-09T16:42:55.654 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:42:55.531+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4355.0:120051 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5447593184 gen 1] snapc 0=[] ondisk+write+known_if_redirected e372) 2019-12-09T16:42:56.812 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:42:56.597+0000 20e25700 -1 RGWWatcher::handle_error cookie 5447593184 err (107) Transport endpoint is not connected 2019-12-09T16:43:49.777 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:43:49.772+0000 1694f700 -1 osd.0 398 get_health_metrics reporting 38 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e375) 2019-12-09T16:43:50.760 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:43:50.753+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 111 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e375) 2019-12-09T16:43:51.821 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:43:51.808+0000 1694f700 -1 osd.0 400 get_health_metrics reporting 158 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e375) 2019-12-09T16:43:52.902 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:43:52.870+0000 1694f700 -1 osd.0 401 get_health_metrics reporting 158 slow ops, oldest is osd_op(client.4361.0:92086 7.d 7.c157980d (undecoded) ondisk+retry+read+known_if_redirected e375) 2019-12-09T16:44:04.288 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:31:40.552 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:04.291 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:31:40.556 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:04.307 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:31:40.529 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:04.312 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:31:40.534 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:04.324 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:31:40.546 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:04.332 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:31:40.549 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:04.611 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:31:40.875 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:04.728 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:31:40.948 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:04.736 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:31:40.958 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:45:04.312 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:04.308+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:05.304 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:05.299+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:06.029 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:06.022+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:06.264 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:06.260+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:06.758 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:06.752+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:06.996 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:06.985+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:07.025 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:07.019+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:07.271 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:07.265+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:07.291 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:07.285+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:07.739 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:07.734+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:07.796 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:07.791+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:08.018 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:08.013+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:08.057 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:08.050+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:08.057 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:08.053+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:08.291 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:08.286+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:08.321 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:08.315+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:08.717 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:08.712+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:08.784 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:08.780+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:09.012 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:09.007+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:09.065 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:09.059+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:09.065 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:09.061+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:09.267 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:09.262+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:09.354 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:09.349+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:09.754 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:09.748+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:09.777 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:09.771+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:09.987 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:09.979+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:10.096 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:10.089+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:10.097 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:10.092+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:10.290 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:10.285+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:10.396 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:10.390+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:10.739 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:10.733+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:10.787 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:10.779+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:10.968 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:10.962+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:11.097 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:11.092+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:11.098 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:11.094+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:11.305 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:11.298+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:11.438 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:11.431+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:11.705 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:11.695+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:11.818 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:11.811+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:11.953 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:11.948+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:12.071 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:12.064+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:12.072 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:12.067+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:12.349 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:12.343+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:12.409 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:12.403+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:12.675 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:12.670+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:12.867 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:12.861+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:12.991 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:12.986+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:13.030 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:13.025+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:13.031 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:13.026+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:13.368 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:13.362+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:13.439 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:13.432+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:13.636 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:13.630+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:13.830 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:13.825+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:13.972 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:13.965+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:14.022 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:14.016+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:14.023 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:14.018+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:14.384 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:14.379+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:14.463 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:14.457+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:14.596 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:14.590+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:14.878 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:14.871+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:14.934 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:14.929+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:15.037 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:15.032+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:15.038 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:15.034+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:15.359 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:15.355+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:15.426 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:15.420+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:15.617 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:15.610+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:15.899 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:15.892+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:15.987 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:15.982+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:16.008 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:16.002+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:16.009 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:16.005+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:16.401 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:16.395+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:16.471 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:16.464+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:16.656 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:16.650+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:16.893 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:16.886+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:17.005 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:16.999+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:17.023 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:17.016+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:17.023 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:17.019+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:17.391 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:17.385+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:17.478 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:17.471+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:17.628 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:17.621+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:17.907 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:17.900+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:17.984 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:17.978+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:17.985 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:17.980+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:18.056 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:18.051+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:18.429 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:18.423+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:18.503 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:18.498+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:18.659 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:18.653+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:18.890 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:18.884+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:18.941 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:18.934+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:18.942 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:18.938+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:19.069 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:19.063+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:19.416 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:19.410+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:19.548 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:19.542+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:19.631 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:19.624+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:19.893 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:19.887+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:19.980 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:19.975+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:19.981 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:19.977+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:20.034 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:20.028+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:20.425 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:20.420+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:20.588 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:20.581+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:20.632 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:20.627+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:20.934 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:20.925+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:20.987 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:20.982+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:20.987 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:20.983+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:21.077 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:21.072+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:21.413 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:21.408+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:21.550 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:21.544+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:21.648 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:21.641+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:21.957 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:21.951+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:21.973 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:21.968+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:21.973 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:21.969+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:22.077 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:22.071+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:22.426 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:22.421+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:22.588 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:22.580+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:22.637 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:22.630+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:22.914 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:22.907+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:23.002 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:22.996+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:23.002 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:22.999+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:23.071 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:23.066+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:23.439 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:23.433+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:23.588 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:23.579+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:23.634 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:23.627+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:23.905 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:23.898+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:24.056 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:24.049+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:24.056 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:24.052+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:24.098 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:24.093+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:24.413 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:24.407+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:24.623 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:24.616+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:24.639 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:24.632+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:24.901 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:24.889+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:25.013 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:25.007+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:25.014 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:25.010+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:25.082 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:25.076+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:25.458 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:25.452+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:25.629 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:25.622+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:25.671 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:25.665+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:25.855 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:25.849+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:25.980 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:25.974+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:25.982 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:25.977+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:26.091 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:26.085+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:26.469 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:26.464+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:26.626 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:26.620+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:26.662 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:26.655+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:26.831 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:26.824+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:26.981 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:26.975+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:26.981 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:26.977+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:27.093 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:27.088+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:27.510 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:27.505+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:27.617 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:27.610+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:27.632 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:27.626+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:27.858 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:27.851+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:27.937 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:27.930+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:27.938 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:27.933+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:28.142 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:28.137+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:28.507 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:28.501+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:28.600 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:28.594+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:28.646 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:28.641+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:28.876 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:28.867+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:28.894 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:28.887+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:28.895 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:28.890+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:29.170 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:29.165+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:29.518 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:29.513+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:29.612 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:29.605+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:29.650 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:33:05.914 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:45:29.652 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:29.647+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:29.659 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:33:05.923 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:45:29.705 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:33:05.927 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:45:29.707 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:33:05.929 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:45:29.715 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:33:05.937 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:45:29.717 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:33:05.939 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:45:29.798 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:33:06.019 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:45:29.825 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:33:06.047 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:45:29.863 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:29.856+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:29.885 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:29.880+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:29.885 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:29.881+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:29.924 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:33:06.188 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:45:30.216 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:30.212+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:30.543 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:30.538+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:30.627 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:30.621+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:30.630 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:30.624+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:30.892 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:30.885+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:30.902 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:30.896+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:30.903 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:30.898+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:31.232 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:31.226+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:31.515 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:31.510+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:31.641 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:31.635+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:31.669 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:31.663+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:31.895 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:31.890+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:31.908 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:31.900+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:31.909 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:31.905+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:32.255 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:32.250+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:32.499 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:32.494+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:32.677 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:32.671+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:32.697 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:32.690+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:32.859 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:32.852+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:32.963 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:32.956+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:32.964 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:32.959+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:33.269 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:33.264+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:33.526 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:33.521+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:33.653 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:33.647+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:33.671 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:33.665+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:33.815 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:33.809+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:33.994 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:33.988+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:33.995 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:33.991+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:34.290 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:34.283+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:34.510 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:34.505+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:34.633 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:34.626+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:34.690 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:34.683+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:34.855 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:34.849+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:35.016 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:35.009+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:35.017 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:35.012+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:35.332 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:35.327+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:35.497 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:35.490+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:35.638 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:35.631+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:35.648 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:35.641+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:35.912 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:35.903+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:35.988 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:35.979+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:35.989 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:35.983+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:36.328 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:36.323+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:36.524 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:36.517+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:36.646 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:36.639+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:36.657 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:36.651+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:36.899 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:36.892+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:37.036 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:37.030+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:37.037 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:37.032+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:37.352 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:37.346+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:37.478 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:37.472+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:37.622 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:37.615+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:37.654 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:37.648+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:37.876 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:37.870+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:38.045 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:38.035+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:38.047 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:38.039+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:38.333 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:38.326+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:38.499 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:38.492+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:38.654 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:38.647+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:38.697 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:38.690+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:38.924 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:38.917+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:39.013 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:39.007+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:39.015 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:39.009+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:39.301 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:39.297+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:39.561 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:39.557+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:39.591 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:45:39.580+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 47sec e417 v417) 2019-12-09T16:45:39.628 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:39.620+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:39.750 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:39.744+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:39.910 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:39.904+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:39.984 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:39.978+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:39.985 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:39.981+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:40.299 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:40.293+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:40.560 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:40.555+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:40.629 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:40.618+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:40.797 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:40.790+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:40.944 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:40.938+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:40.957 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:40.950+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:40.958 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:40.953+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:41.336 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:41.330+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:41.604 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:41.599+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:41.632 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:41.625+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:41.804 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:41.797+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:41.931 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:41.925+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:41.932 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:41.927+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:41.957 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:41.950+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:42.322 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:42.317+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:42.591 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:42.585+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:42.599 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:42.592+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:42.857 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:42.850+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:42.962 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:42.957+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:42.963 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:42.959+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:43.010 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:43.003+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:43.323 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:43.318+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:43.581 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:43.575+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:43.614 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:43.608+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:43.833 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:43.826+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:43.934 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:43.928+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:43.935 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:43.930+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:43.986 INFO:tasks.ceph.osd.7.smithi022.stderr:2019-12-09T16:45:43.980+0000 1694f700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.205992+0000 front 2019-12-09T16:44:22.214720+0000 (oldest deadline 2019-12-09T16:45:06.207524+0000) 2019-12-09T16:45:44.366 INFO:tasks.ceph.osd.2.smithi035.stderr:2019-12-09T16:45:44.361+0000 1694f700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:22.678085+0000 front 2019-12-09T16:44:23.346801+0000 (oldest deadline 2019-12-09T16:45:05.421227+0000) 2019-12-09T16:45:44.562 INFO:tasks.ceph.osd.3.smithi035.stderr:2019-12-09T16:45:44.556+0000 1694f700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.344647+0000 front 2019-12-09T16:44:22.480424+0000 (oldest deadline 2019-12-09T16:45:07.133036+0000) 2019-12-09T16:45:44.651 INFO:tasks.ceph.osd.5.smithi022.stderr:2019-12-09T16:45:44.645+0000 1694f700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:18.228513+0000 front 2019-12-09T16:44:23.929542+0000 (oldest deadline 2019-12-09T16:45:03.780141+0000) 2019-12-09T16:45:44.675 INFO:tasks.ceph.mon.b.smithi022.stderr:2019-12-09T16:45:44.668+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.35:6800/9790,v1:172.21.15.35:6801/9790] for 47sec e417 v417) 2019-12-09T16:45:44.814 INFO:tasks.ceph.osd.4.smithi022.stderr:2019-12-09T16:45:44.808+0000 1694f700 -1 osd.4 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:21.935135+0000 front 2019-12-09T16:44:22.229503+0000 (oldest deadline 2019-12-09T16:45:07.195620+0000) 2019-12-09T16:45:44.889 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:44.884+0000 1694f700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.35:6804 osd.1 since back 2019-12-09T16:44:23.939935+0000 front 2019-12-09T16:44:24.283055+0000 (oldest deadline 2019-12-09T16:45:06.749181+0000) 2019-12-09T16:45:44.890 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:44.886+0000 1694f700 -1 osd.0 417 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:46.048 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:45.982+0000 1694f700 -1 osd.0 418 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:47.159 INFO:tasks.ceph.osd.0.smithi035.stderr:2019-12-09T16:45:47.150+0000 1694f700 -1 osd.0 419 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4355.0:120335 5.5 5:a93a5511:::notify.2:head [watch ping cookie 3472613360 gen 1] snapc 0=[] ondisk+write+known_if_redirected e417) 2019-12-09T16:45:48.268 INFO:tasks.rgw.client.0.smithi035.stdout:2019-12-09T16:45:47.914+0000 20e25700 -1 RGWWatcher::handle_error cookie 3472613360 err (107) Transport endpoint is not connected 2019-12-09T16:46:05.219 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:33:41.483 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:46:05.247 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:33:41.511 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:46:05.286 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:33:41.508 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:46:05.289 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:33:41.510 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:46:05.290 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:33:41.512 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:46:05.305 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:33:41.526 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:46:05.355 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:33:41.620 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:46:05.373 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:33:41.592 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:46:05.385 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:33:41.607 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:46:48.840 INFO:tasks.ceph.osd.1.smithi035.stderr:*** Caught signal (Aborted) ** 2019-12-09T16:46:48.840 INFO:tasks.ceph.osd.1.smithi035.stderr: in thread 3a280700 thread_name:tp_osd_tp 2019-12-09T16:46:48.911 INFO:tasks.ceph.osd.1.smithi035.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T16:46:48.911 INFO:tasks.ceph.osd.1.smithi035.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T16:46:48.911 INFO:tasks.ceph.osd.1.smithi035.stderr: 2: (pthread_kill()+0x31) [0xc09ba61] 2019-12-09T16:46:48.911 INFO:tasks.ceph.osd.1.smithi035.stderr: 3: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d const*, char const*, unsigned long)+0x244) [0xcb33a4] 2019-12-09T16:46:48.911 INFO:tasks.ceph.osd.1.smithi035.stderr: 4: (ceph::HeartbeatMap::reset_timeout(ceph::heartbeat_handle_d*, unsigned long, unsigned long)+0x252) [0xcb3972] 2019-12-09T16:46:48.911 INFO:tasks.ceph.osd.1.smithi035.stderr: 5: (OSD::advance_pg(unsigned int, PG*, ThreadPool::TPHandle&, PeeringCtx&)+0x613) [0x703343] 2019-12-09T16:46:48.912 INFO:tasks.ceph.osd.1.smithi035.stderr: 6: (OSD::dequeue_peering_evt(OSDShard*, PG*, std::shared_ptr, ThreadPool::TPHandle&)+0x9a) [0x704b2a] 2019-12-09T16:46:48.912 INFO:tasks.ceph.osd.1.smithi035.stderr: 7: (ceph::osd::scheduler::PGPeeringItem::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0x52) [0x915072] 2019-12-09T16:46:48.912 INFO:tasks.ceph.osd.1.smithi035.stderr: 8: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x945) [0x6f8075] 2019-12-09T16:46:48.912 INFO:tasks.ceph.osd.1.smithi035.stderr: 9: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x45a) [0xcd228a] 2019-12-09T16:46:48.912 INFO:tasks.ceph.osd.1.smithi035.stderr: 10: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0xcda690] 2019-12-09T16:46:48.912 INFO:tasks.ceph.osd.1.smithi035.stderr: 11: (()+0x7e65) [0xc096e65] 2019-12-09T16:46:48.912 INFO:tasks.ceph.osd.1.smithi035.stderr: 12: (clone()+0x6d) [0xd20e88d] 2019-12-09T16:46:48.935 INFO:tasks.ceph.osd.1.smithi035.stderr:2019-12-09T16:46:48.929+0000 3a280700 -1 *** Caught signal (Aborted) ** 2019-12-09T16:46:48.935 INFO:tasks.ceph.osd.1.smithi035.stderr: in thread 3a280700 thread_name:tp_osd_tp 2019-12-09T16:46:48.935 INFO:tasks.ceph.osd.1.smithi035.stderr: 2019-12-09T16:46:48.935 INFO:tasks.ceph.osd.1.smithi035.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T16:46:48.935 INFO:tasks.ceph.osd.1.smithi035.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T16:46:48.936 INFO:tasks.ceph.osd.1.smithi035.stderr: 2: (pthread_kill()+0x31) [0xc09ba61] 2019-12-09T16:46:48.936 INFO:tasks.ceph.osd.1.smithi035.stderr: 3: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d const*, char const*, unsigned long)+0x244) [0xcb33a4] 2019-12-09T16:46:48.936 INFO:tasks.ceph.osd.1.smithi035.stderr: 4: (ceph::HeartbeatMap::reset_timeout(ceph::heartbeat_handle_d*, unsigned long, unsigned long)+0x252) [0xcb3972] 2019-12-09T16:46:48.936 INFO:tasks.ceph.osd.1.smithi035.stderr: 5: (OSD::advance_pg(unsigned int, PG*, ThreadPool::TPHandle&, PeeringCtx&)+0x613) [0x703343] 2019-12-09T16:46:48.936 INFO:tasks.ceph.osd.1.smithi035.stderr: 6: (OSD::dequeue_peering_evt(OSDShard*, PG*, std::shared_ptr, ThreadPool::TPHandle&)+0x9a) [0x704b2a] 2019-12-09T16:46:48.936 INFO:tasks.ceph.osd.1.smithi035.stderr: 7: (ceph::osd::scheduler::PGPeeringItem::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0x52) [0x915072] 2019-12-09T16:46:48.936 INFO:tasks.ceph.osd.1.smithi035.stderr: 8: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x945) [0x6f8075] 2019-12-09T16:46:48.937 INFO:tasks.ceph.osd.1.smithi035.stderr: 9: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x45a) [0xcd228a] 2019-12-09T16:46:48.937 INFO:tasks.ceph.osd.1.smithi035.stderr: 10: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0xcda690] 2019-12-09T16:46:48.937 INFO:tasks.ceph.osd.1.smithi035.stderr: 11: (()+0x7e65) [0xc096e65] 2019-12-09T16:46:48.937 INFO:tasks.ceph.osd.1.smithi035.stderr: 12: (clone()+0x6d) [0xd20e88d] 2019-12-09T16:46:48.937 INFO:tasks.ceph.osd.1.smithi035.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T16:46:48.937 INFO:tasks.ceph.osd.1.smithi035.stderr: 2019-12-09T16:46:49.316 INFO:tasks.ceph.osd.1.smithi035.stderr: 0> 2019-12-09T16:46:48.929+0000 3a280700 -1 *** Caught signal (Aborted) ** 2019-12-09T16:46:49.317 INFO:tasks.ceph.osd.1.smithi035.stderr: in thread 3a280700 thread_name:tp_osd_tp 2019-12-09T16:46:49.317 INFO:tasks.ceph.osd.1.smithi035.stderr: 2019-12-09T16:46:49.317 INFO:tasks.ceph.osd.1.smithi035.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T16:46:49.317 INFO:tasks.ceph.osd.1.smithi035.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T16:46:49.317 INFO:tasks.ceph.osd.1.smithi035.stderr: 2: (pthread_kill()+0x31) [0xc09ba61] 2019-12-09T16:46:49.317 INFO:tasks.ceph.osd.1.smithi035.stderr: 3: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d const*, char const*, unsigned long)+0x244) [0xcb33a4] 2019-12-09T16:46:49.318 INFO:tasks.ceph.osd.1.smithi035.stderr: 4: (ceph::HeartbeatMap::reset_timeout(ceph::heartbeat_handle_d*, unsigned long, unsigned long)+0x252) [0xcb3972] 2019-12-09T16:46:49.318 INFO:tasks.ceph.osd.1.smithi035.stderr: 5: (OSD::advance_pg(unsigned int, PG*, ThreadPool::TPHandle&, PeeringCtx&)+0x613) [0x703343] 2019-12-09T16:46:49.318 INFO:tasks.ceph.osd.1.smithi035.stderr: 6: (OSD::dequeue_peering_evt(OSDShard*, PG*, std::shared_ptr, ThreadPool::TPHandle&)+0x9a) [0x704b2a] 2019-12-09T16:46:49.318 INFO:tasks.ceph.osd.1.smithi035.stderr: 7: (ceph::osd::scheduler::PGPeeringItem::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0x52) [0x915072] 2019-12-09T16:46:49.318 INFO:tasks.ceph.osd.1.smithi035.stderr: 8: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x945) [0x6f8075] 2019-12-09T16:46:49.318 INFO:tasks.ceph.osd.1.smithi035.stderr: 9: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x45a) [0xcd228a] 2019-12-09T16:46:49.318 INFO:tasks.ceph.osd.1.smithi035.stderr: 10: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0xcda690] 2019-12-09T16:46:49.319 INFO:tasks.ceph.osd.1.smithi035.stderr: 11: (()+0x7e65) [0xc096e65] 2019-12-09T16:46:49.319 INFO:tasks.ceph.osd.1.smithi035.stderr: 12: (clone()+0x6d) [0xd20e88d] 2019-12-09T16:46:49.319 INFO:tasks.ceph.osd.1.smithi035.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T16:46:49.319 INFO:tasks.ceph.osd.1.smithi035.stderr: 2019-12-09T16:46:49.687 INFO:tasks.ceph.osd.1.smithi035.stderr: -17> 2019-12-09T16:46:48.929+0000 3a280700 -1 *** Caught signal (Aborted) ** 2019-12-09T16:46:49.687 INFO:tasks.ceph.osd.1.smithi035.stderr: in thread 3a280700 thread_name:tp_osd_tp 2019-12-09T16:46:49.687 INFO:tasks.ceph.osd.1.smithi035.stderr: 2019-12-09T16:46:49.688 INFO:tasks.ceph.osd.1.smithi035.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T16:46:49.688 INFO:tasks.ceph.osd.1.smithi035.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T16:46:49.688 INFO:tasks.ceph.osd.1.smithi035.stderr: 2: (pthread_kill()+0x31) [0xc09ba61] 2019-12-09T16:46:49.688 INFO:tasks.ceph.osd.1.smithi035.stderr: 3: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d const*, char const*, unsigned long)+0x244) [0xcb33a4] 2019-12-09T16:46:49.688 INFO:tasks.ceph.osd.1.smithi035.stderr: 4: (ceph::HeartbeatMap::reset_timeout(ceph::heartbeat_handle_d*, unsigned long, unsigned long)+0x252) [0xcb3972] 2019-12-09T16:46:49.688 INFO:tasks.ceph.osd.1.smithi035.stderr: 5: (OSD::advance_pg(unsigned int, PG*, ThreadPool::TPHandle&, PeeringCtx&)+0x613) [0x703343] 2019-12-09T16:46:49.689 INFO:tasks.ceph.osd.1.smithi035.stderr: 6: (OSD::dequeue_peering_evt(OSDShard*, PG*, std::shared_ptr, ThreadPool::TPHandle&)+0x9a) [0x704b2a] 2019-12-09T16:46:49.689 INFO:tasks.ceph.osd.1.smithi035.stderr: 7: (ceph::osd::scheduler::PGPeeringItem::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0x52) [0x915072] 2019-12-09T16:46:49.689 INFO:tasks.ceph.osd.1.smithi035.stderr: 8: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x945) [0x6f8075] 2019-12-09T16:46:49.689 INFO:tasks.ceph.osd.1.smithi035.stderr: 9: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x45a) [0xcd228a] 2019-12-09T16:46:49.689 INFO:tasks.ceph.osd.1.smithi035.stderr: 10: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0xcda690] 2019-12-09T16:46:49.689 INFO:tasks.ceph.osd.1.smithi035.stderr: 11: (()+0x7e65) [0xc096e65] 2019-12-09T16:46:49.689 INFO:tasks.ceph.osd.1.smithi035.stderr: 12: (clone()+0x6d) [0xd20e88d] 2019-12-09T16:46:49.689 INFO:tasks.ceph.osd.1.smithi035.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T16:46:49.690 INFO:tasks.ceph.osd.1.smithi035.stderr: 2019-12-09T16:46:52.860 INFO:tasks.ceph.osd.1.smithi035.stderr:daemon-helper: command crashed with signal 6 2019-12-09T16:48:28.426 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:36:04.690 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:28.454 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:36:04.676 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:28.454 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:36:04.676 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:28.459 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:36:04.723 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:28.461 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:36:04.683 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:28.469 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:36:04.691 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:28.639 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:36:04.904 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:28.670 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:36:04.892 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:28.673 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:36:04.895 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:37.137 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:41:13.402 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:37.167 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:41:13.380 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:37.167 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:41:13.383 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:37.167 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:41:13.411 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:37.168 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:41:13.390 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:37.173 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:41:13.395 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:37.399 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:41:13.664 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:37.436 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:41:13.658 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:37.465 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:41:13.687 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:52.263 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:41:28.527 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:52.271 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:41:28.536 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:52.284 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:41:28.506 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:52.285 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:41:28.506 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:52.293 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:41:28.515 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:52.311 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:41:28.533 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:52.319 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:41:28.584 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:52.324 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:41:28.546 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:52.345 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:41:28.567 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:57:32.110 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:45:08.375 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:57:32.131 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:45:08.353 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:57:32.139 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:45:08.362 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:57:32.143 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:45:08.366 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:57:32.156 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:45:08.378 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:57:32.160 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:45:08.426 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T16:57:32.437 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:45:08.659 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T16:57:32.451 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:45:08.673 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T16:57:32.640 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:45:08.905 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:14.722 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:49:50.987 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:14.742 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:49:51.008 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:14.817 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:49:51.039 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:14.818 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:49:51.040 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:14.819 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:49:51.042 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:14.821 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:49:51.043 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:14.947 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:49:51.212 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:14.955 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:49:51.177 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:14.973 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:49:51.189 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:42.264 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:50:18.529 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:42.287 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:50:18.552 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:42.292 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:50:18.514 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:42.293 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:50:18.515 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:42.297 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:50:18.519 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:42.301 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:50:18.523 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:42.374 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:50:18.596 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:42.385 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:50:18.607 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:02:42.403 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:50:18.668 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:54.845 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:55:31.110 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:54.859 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:55:31.124 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:54.869 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:55:31.092 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:54.875 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:55:31.097 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:54.876 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:55:31.098 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:54.883 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:55:31.105 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:55.157 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:55:31.423 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:55.175 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:55:31.397 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:55.258 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:55:31.480 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:37.157 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:58:13.423 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:37.162 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:58:13.428 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:37.216 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:58:13.438 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:37.223 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:58:13.444 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:37.223 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:58:13.444 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:37.224 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:58:13.445 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:37.262 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:07:58:13.528 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:37.296 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:07:58:13.519 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:37.328 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:07:58:13.550 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:03.955 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:00:40.220 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:03.964 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:00:40.230 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:04.051 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:00:40.273 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:04.052 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:00:40.275 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:04.059 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:00:40.281 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:04.065 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:00:40.288 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:04.162 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:00:40.427 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:04.217 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:00:40.440 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:04.224 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:00:40.447 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:15.359 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:05:51.625 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:15.373 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:05:51.634 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:15.394 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:05:51.616 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:15.397 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:05:51.620 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:15.405 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:05:51.628 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:15.408 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:05:51.631 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:15.621 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:05:51.844 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:15.651 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:05:51.874 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:15.725 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:05:51.991 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:19:04.319 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:06:40.585 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:19:04.379 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:06:40.645 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:19:04.382 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:06:40.605 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:19:04.391 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:06:40.614 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:19:04.398 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:06:40.621 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:19:04.402 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:06:40.625 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:19:04.455 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:06:40.677 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:19:04.464 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:06:40.683 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:19:04.474 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:06:40.740 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:14.968 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:07:51.234 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:15.027 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:07:51.293 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:15.058 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:07:51.280 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:15.091 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:07:51.314 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:15.161 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:07:51.384 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:15.165 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:07:51.388 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:15.169 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:07:51.392 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:15.225 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:07:51.448 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:15.236 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:07:51.502 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:23:25.677 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:11:01.943 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:23:25.688 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:11:01.954 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:23:25.713 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:11:01.936 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:23:25.714 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:11:01.937 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:23:25.720 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:11:01.944 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:23:25.723 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:11:01.947 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:23:25.898 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:11:02.121 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:23:25.910 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:11:02.133 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:23:25.943 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:11:02.209 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:27:27.762 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:15:04.028 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:27:27.785 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:15:04.042 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:27:27.852 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:15:04.075 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:27:27.858 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:15:04.080 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:27:27.860 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:15:04.084 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:27:27.862 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:15:04.086 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:27:27.960 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:15:04.226 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:27:27.986 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:15:04.209 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:27:27.992 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:15:04.215 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:28:37.856 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:16:14.122 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:28:37.875 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:16:14.098 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:28:37.876 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:16:14.099 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:28:37.878 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:16:14.102 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:28:37.879 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:16:14.103 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:28:37.881 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:16:14.148 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:28:37.956 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:16:14.179 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:28:37.958 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:16:14.181 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:28:38.013 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:16:14.279 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:33:50.871 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:21:27.136 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:33:50.897 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:21:27.120 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:33:50.901 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:21:27.167 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:33:50.902 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:21:27.125 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:33:50.912 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:21:27.136 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:33:50.916 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:21:27.139 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:33:51.125 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:21:27.348 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:33:51.193 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:21:27.459 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:33:51.236 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:21:27.459 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:50.442 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:23:26.709 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:50.462 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:23:26.713 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:50.516 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:23:26.740 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:50.518 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:23:26.742 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:50.519 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:23:26.743 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:50.535 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:23:26.759 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:50.573 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:23:26.840 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:50.607 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:23:26.831 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:50.609 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:23:26.833 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:39:01.405 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:26:37.671 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:39:01.423 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:26:37.678 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:39:01.426 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:26:37.650 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:39:01.432 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:26:37.655 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:39:01.432 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:26:37.656 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:39:01.441 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:26:37.665 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:39:01.596 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:26:37.820 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:39:01.615 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:26:37.838 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:39:01.634 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:26:37.900 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:12.580 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:31:48.846 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:12.622 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:31:48.888 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:12.781 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:31:49.004 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:12.784 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:31:49.007 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:12.792 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:31:49.015 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:12.794 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:31:49.017 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:12.861 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:31:49.127 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:12.920 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:31:49.143 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:12.970 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:31:49.194 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:14.627 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:31:50.849 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:14.628 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:31:50.851 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:14.665 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:31:50.888 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:14.684 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:31:50.907 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:14.685 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:31:50.909 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:14.696 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:31:50.919 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:15.065 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:31:51.330 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:15.066 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:31:51.332 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:44:15.083 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:31:51.349 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:26.998 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:37:03.264 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:27.022 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:37:03.287 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:27.033 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:37:03.256 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:27.037 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:37:03.260 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:27.048 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:37:03.271 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:27.055 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:37:03.278 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:27.288 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:37:03.553 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:27.315 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:37:03.538 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:27.339 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:37:03.562 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:52:34.687 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:40:10.952 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:52:34.696 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:40:10.962 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:52:34.732 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:40:10.955 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:52:34.750 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:40:10.973 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:52:34.791 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:40:11.013 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:52:34.797 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:40:11.016 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:52:34.817 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:40:11.083 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:52:34.879 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:40:11.102 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:52:34.911 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:40:11.131 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:39.275 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:42:15.540 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:39.280 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:42:15.546 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:39.287 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:42:15.510 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:39.290 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:42:15.513 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:39.291 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:42:15.513 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:39.294 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:42:15.517 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:39.390 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:42:15.612 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:39.392 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:42:15.614 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:39.454 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:42:15.720 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.225 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:47:27.490 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.249 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:47:27.471 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.249 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:47:27.504 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.254 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:47:27.477 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.261 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:47:27.483 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.270 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:47:27.493 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.496 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:47:27.761 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.499 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:47:27.722 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.519 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:47:27.742 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:00:57.442 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:48:33.707 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:00:57.511 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:48:33.776 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:00:57.540 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:48:33.763 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:00:57.561 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:48:33.784 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:00:57.568 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:48:33.829 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:00:57.572 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:48:33.795 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:00:57.580 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:48:33.803 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:00:57.635 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:48:33.857 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:00:57.645 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:48:33.867 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:12.961 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:48:49.226 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:12.972 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:48:49.237 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:13.027 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:48:49.292 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:13.030 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:48:49.253 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:13.044 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:48:49.267 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:13.075 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:48:49.297 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:13.076 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:48:49.299 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:13.101 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:48:49.323 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:13.106 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:48:49.329 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:05:01.206 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:52:37.471 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:05:01.223 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:52:37.489 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:05:01.235 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:52:37.457 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:05:01.239 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:52:37.462 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:05:01.243 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:52:37.466 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:05:01.256 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:52:37.479 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:05:01.470 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:52:37.692 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:05:01.483 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:52:37.705 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:05:01.528 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:52:37.794 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:20.113 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:56:56.379 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:20.127 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:56:56.392 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:20.143 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:56:56.365 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:20.179 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:56:56.402 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:20.181 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:56:56.404 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:20.189 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:56:56.411 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:20.283 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:56:56.549 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:20.303 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:56:56.525 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:20.305 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:56:56.528 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:10:11.488 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:57:47.753 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:10:11.508 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:57:47.772 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:10:11.511 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:57:47.733 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:10:11.511 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:57:47.734 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:10:11.513 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:57:47.736 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:10:11.515 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:57:47.738 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:10:11.569 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:08:57:47.792 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:10:11.573 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:08:57:47.795 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:10:11.642 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:08:57:47.907 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:24.199 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:03:00.465 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:24.215 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:03:00.480 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:24.227 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:03:00.450 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:24.232 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:03:00.454 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:24.238 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:03:00.461 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:24.246 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:03:00.468 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:24.527 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:03:00.749 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:24.529 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:03:00.751 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:24.543 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:03:00.809 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.330 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:05:18.595 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.351 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:05:18.574 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.385 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:05:18.650 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.399 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:05:18.622 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.403 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:05:18.626 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.416 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:05:18.639 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.495 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:05:18.717 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.495 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:05:18.717 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.570 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:05:18.833 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:38.295 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:08:14.561 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:38.319 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:08:14.537 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:38.319 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:08:14.573 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:38.320 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:08:14.543 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:38.322 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:08:14.545 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:38.330 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:08:14.553 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:38.468 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:08:14.691 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:38.473 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:08:14.696 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:38.498 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:08:14.763 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:25:49.449 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:13:25.714 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:25:49.476 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:13:25.742 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:25:49.479 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:13:25.702 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:25:49.481 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:13:25.704 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:25:49.489 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:13:25.712 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:25:49.489 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:13:25.712 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:25:49.722 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:13:25.945 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:25:49.724 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:13:25.947 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:25:49.765 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:13:26.031 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:04.610 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:13:40.875 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:04.627 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:13:40.850 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:04.640 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:13:40.905 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:04.643 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:13:40.866 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:04.649 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:13:40.872 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:04.660 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:13:40.884 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:04.675 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:13:40.898 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:04.685 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:13:40.951 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:04.698 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:13:40.921 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:00.899 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:18:37.161 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:00.917 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:18:37.169 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:00.919 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:18:37.142 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:00.924 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:18:37.147 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:00.926 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:18:37.149 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:00.929 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:18:37.152 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:01.112 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:18:37.378 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:01.114 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:18:37.337 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:01.128 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:18:37.351 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:26.693 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:22:02.959 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:26.703 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:22:02.969 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:26.741 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:22:02.964 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:26.748 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:22:02.971 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:26.754 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:22:02.976 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:26.758 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:22:02.981 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:26.863 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:22:03.085 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:26.895 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:22:03.117 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:26.938 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:22:03.203 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:36:09.204 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:23:45.470 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:36:09.221 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:23:45.486 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:36:09.241 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:23:45.464 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:36:09.242 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:23:45.466 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:36:09.249 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:23:45.472 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:36:09.250 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:23:45.473 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:36:09.359 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:23:45.581 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:36:09.380 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:23:45.603 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:36:09.406 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:23:45.672 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:22.290 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:28:58.555 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:22.305 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:28:58.571 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:22.335 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:28:58.557 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:22.336 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:28:58.559 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:22.339 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:28:58.562 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:22.340 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:28:58.563 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:22.557 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:28:58.779 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:22.569 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:28:58.834 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:22.616 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:28:58.839 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:44.156 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:29:20.422 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:44.187 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:29:20.452 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:44.212 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:29:20.433 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:44.227 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:29:20.450 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:44.242 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:29:20.508 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:44.244 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:29:20.467 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:44.248 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:29:20.471 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:44.267 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:29:20.490 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:44.275 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:29:20.498 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:49.624 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:30:25.890 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:49.630 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:30:25.896 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:49.644 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:30:25.867 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:49.697 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:30:25.920 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:49.709 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:30:25.932 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:49.714 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:30:25.937 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:49.740 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:30:26.005 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:49.766 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:30:25.989 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:49.770 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:30:25.994 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:31.860 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:34:08.126 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:31.881 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:34:08.101 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:31.881 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:34:08.133 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:31.882 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:34:08.104 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:31.882 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:34:08.104 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:31.888 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:34:08.109 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:32.125 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:34:08.347 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:32.134 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:34:08.356 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:32.154 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:34:08.419 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:11.941 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:38:48.206 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:11.961 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:38:48.180 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:11.961 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:38:48.209 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:11.985 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:38:48.208 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:12.001 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:38:48.224 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:12.017 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:38:48.240 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:12.123 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:38:48.346 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:12.179 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:38:48.394 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:12.255 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:38:48.520 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:42.376 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:39:18.641 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:42.390 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:39:18.656 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:42.401 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:39:18.624 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:42.405 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:39:18.628 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:42.412 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:39:18.632 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:42.412 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:39:18.633 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:42.468 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:39:18.691 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:42.475 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:39:18.698 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:42.560 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:39:18.825 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:58.309 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:44:34.574 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:58.335 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:44:34.592 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:58.341 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:44:34.564 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:58.343 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:44:34.566 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:58.348 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:44:34.571 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:58.355 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:44:34.578 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:58.583 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:44:34.806 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:58.583 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:44:34.806 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:58.658 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:44:34.923 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:34.068 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:47:10.333 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:34.071 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:47:10.337 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:34.127 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:47:10.349 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:34.139 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:47:10.362 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:34.143 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:47:10.364 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:34.144 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:47:10.365 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:34.207 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:47:10.472 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:34.253 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:47:10.476 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:34.266 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:47:10.489 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:06.558 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:49:42.822 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:06.569 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:49:42.835 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:06.587 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:49:42.810 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:06.593 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:49:42.816 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:06.597 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:49:42.821 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:06.611 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:49:42.834 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:06.711 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:49:42.935 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:06.761 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:49:42.984 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:06.808 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:49:43.073 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:18.808 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:54:55.073 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:18.815 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:54:55.081 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:18.829 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:54:55.051 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:18.829 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:54:55.052 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:18.836 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:54:55.060 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:18.840 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:54:55.064 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:19.046 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:54:55.269 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:19.065 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:54:55.288 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:19.128 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:54:55.394 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:56.358 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:55:32.624 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:56.442 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:55:32.708 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:56.458 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:55:32.681 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:56.465 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:55:32.689 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:56.482 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:55:32.705 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:56.488 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:55:32.711 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:56.501 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:09:55:32.721 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:56.538 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:09:55:32.761 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:56.581 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:09:55:32.847 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:12:31.178 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:00:07.443 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:12:31.187 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:00:07.453 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:12:31.211 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:00:07.434 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:12:31.215 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:00:07.439 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:12:31.225 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:00:07.448 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:12:31.226 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:00:07.449 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:12:31.439 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:00:07.663 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:12:31.446 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:00:07.669 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:12:31.471 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:00:07.737 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:16:18.811 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:03:55.077 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:16:18.851 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:03:55.117 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:16:18.900 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:03:55.123 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:16:18.901 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:03:55.125 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:16:18.909 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:03:55.132 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:16:18.912 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:03:55.135 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:16:19.020 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:03:55.243 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:16:19.044 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:03:55.262 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:16:19.132 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:03:55.398 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:41.408 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:05:17.673 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:41.415 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:05:17.681 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:41.437 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:05:17.660 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:41.442 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:05:17.666 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:41.520 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:05:17.743 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:41.521 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:05:17.787 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:41.531 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:05:17.754 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:41.571 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:05:17.794 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:41.621 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:05:17.844 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:25.762 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:10:02.028 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:25.782 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:10:02.031 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:25.847 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:10:02.071 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:25.849 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:10:02.072 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:25.849 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:10:02.073 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:25.852 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:10:02.075 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:26.027 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:10:02.249 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:26.027 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:10:02.250 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:26.129 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:10:02.394 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:52.143 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:10:28.408 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:52.160 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:10:28.426 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:52.165 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:10:28.389 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:52.175 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:10:28.394 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:52.179 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:10:28.403 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:52.186 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:10:28.410 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:52.261 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:10:28.485 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:52.275 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:10:28.540 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:52.285 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:10:28.508 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:24:41.310 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:12:17.576 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:24:41.320 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:12:17.587 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:24:41.392 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:12:17.615 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:24:41.394 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:12:17.618 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:24:41.482 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:12:17.748 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:24:41.504 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:12:17.727 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:24:41.524 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:12:17.747 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:24:41.583 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:12:17.806 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:24:41.659 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:12:17.883 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:28:05.795 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:15:42.061 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:28:05.806 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:15:42.030 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:28:05.809 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:15:42.033 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:28:05.809 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:15:42.075 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:28:05.816 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:15:42.035 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:28:05.816 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:15:42.035 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:28:05.973 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:15:42.196 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:28:06.010 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:15:42.276 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:28:06.011 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:15:42.234 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:03.556 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:20:39.821 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:03.618 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:20:39.842 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:03.620 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:20:39.844 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:03.624 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:20:39.891 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:03.791 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:20:40.057 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:03.823 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:20:40.047 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:03.834 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:20:40.058 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:03.836 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:20:40.060 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:04.028 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:20:40.251 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:18.190 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:20:54.456 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:18.201 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:20:54.467 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:18.222 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:20:54.446 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:18.224 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:20:54.448 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:18.231 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:20:54.455 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:18.233 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:20:54.456 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:18.278 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:20:54.502 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:18.284 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:20:54.508 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:18.300 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:20:54.566 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:31.267 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:26:07.533 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:31.298 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:26:07.564 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:31.308 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:26:07.532 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:31.321 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:26:07.537 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:31.321 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:26:07.540 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:31.323 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:26:07.547 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:31.523 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:26:07.730 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:31.552 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:26:07.775 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:31.590 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:26:07.857 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:26.045 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:29:02.305 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:26.073 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:29:02.313 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:26.136 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:29:02.359 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:26.137 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:29:02.361 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:26.138 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:29:02.362 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:26.140 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:29:02.364 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:26.157 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:29:02.423 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:26.297 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:29:02.517 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:26.297 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:29:02.517 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:43:41.892 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:31:18.152 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:43:41.915 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:31:18.131 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:43:41.915 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:31:18.137 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:43:41.915 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:31:18.136 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:43:41.916 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:31:18.171 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:43:41.917 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:31:18.140 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:43:42.026 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:31:18.249 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:43:42.027 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:31:18.251 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:43:42.064 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:31:18.330 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:54.881 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:36:31.147 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:54.900 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:36:31.123 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:54.900 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:36:31.160 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:54.903 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:36:31.127 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:54.904 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:36:31.128 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:54.917 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:36:31.139 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:55.150 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:36:31.416 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:55.217 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:36:31.441 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:55.219 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:36:31.443 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:49:48.353 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:37:24.619 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:49:48.365 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:37:24.628 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:49:48.435 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:37:24.701 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:49:48.438 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:37:24.662 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:49:48.440 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:37:24.664 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:49:48.442 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:37:24.666 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:49:48.448 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:37:24.672 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:49:48.547 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:37:24.771 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:49:48.548 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:37:24.772 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:07.220 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:41:43.487 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:07.245 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:41:43.505 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:07.249 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:41:43.472 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:07.254 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:41:43.478 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:07.256 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:41:43.480 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:07.261 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:41:43.485 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:07.423 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:41:43.647 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:07.449 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:41:43.669 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:07.464 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:41:43.730 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:10.757 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:45:47.023 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:10.783 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:45:47.031 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:10.819 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:45:47.043 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:10.820 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:45:47.044 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:10.846 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:45:47.070 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:10.849 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:45:47.073 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:10.928 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:45:47.152 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:10.942 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:45:47.208 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:11.006 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:45:47.230 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:59:19.603 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:46:55.869 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:59:19.622 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:46:55.845 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:59:19.622 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:46:55.888 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T19:59:19.628 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:46:55.852 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:59:19.634 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:46:55.858 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:59:19.639 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:46:55.863 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:59:19.730 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:46:55.954 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T19:59:19.732 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:46:55.956 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T19:59:19.752 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:46:56.018 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:32.563 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:51:08.829 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:32.586 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:51:08.852 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:32.604 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:51:08.828 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:32.648 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:51:08.871 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:32.656 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:51:08.880 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:32.661 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:51:08.885 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:32.840 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:51:09.106 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:32.949 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:51:09.172 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:32.960 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:51:09.184 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:04:31.469 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:52:07.735 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:04:31.482 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:52:07.748 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:04:31.491 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:52:07.714 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:04:31.492 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:52:07.716 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:04:31.496 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:52:07.720 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:04:31.497 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:52:07.721 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:04:31.573 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:52:07.797 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:04:31.577 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:52:07.801 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:04:31.617 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:52:07.883 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:33.538 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:54:09.794 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:33.554 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:54:09.800 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:33.572 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:54:09.795 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:33.600 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:54:09.823 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:33.607 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:54:09.830 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:33.611 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:54:09.833 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:33.671 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:54:09.895 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:33.680 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:54:09.903 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:33.751 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:54:10.018 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:42.536 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:57:18.801 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:42.593 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:57:18.789 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:42.593 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:57:18.798 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:42.593 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:57:18.792 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:42.593 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:57:18.797 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:42.593 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:57:18.814 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:42.719 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:10:57:18.943 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:42.723 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:10:57:18.947 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:42.731 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:10:57:18.998 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:53.026 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:02:29.288 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:53.047 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:02:29.271 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:53.049 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:02:29.315 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:53.054 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:02:29.278 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:53.057 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:02:29.281 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:53.063 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:02:29.286 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:53.313 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:02:29.537 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:53.359 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:02:29.583 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:53.514 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:02:29.780 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:55.818 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:02:32.084 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:55.827 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:02:32.093 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:55.849 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:02:32.072 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:55.851 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:02:32.075 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:55.855 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:02:32.079 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:55.859 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:02:32.126 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:55.884 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:02:32.108 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:55.901 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:02:32.124 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:55.947 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:02:32.170 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:03.270 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:07:39.536 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:03.297 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:07:39.518 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:03.297 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:07:39.548 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:03.299 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:07:39.523 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:03.307 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:07:39.531 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:03.314 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:07:39.538 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:03.543 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:07:39.767 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:03.547 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:07:39.771 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:03.639 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:07:39.901 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:23:17.957 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:10:54.222 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:23:17.965 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:10:54.229 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:23:18.050 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:10:54.273 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:23:18.053 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:10:54.276 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:23:18.062 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:10:54.285 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:23:18.064 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:10:54.287 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:23:18.121 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:10:54.388 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:23:18.162 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:10:54.386 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:23:18.166 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:10:54.390 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:25:15.167 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:12:51.433 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:25:15.181 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:12:51.447 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:25:15.210 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:12:51.433 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:25:15.223 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:12:51.446 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:25:15.283 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:12:51.507 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:25:15.294 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:12:51.517 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:25:15.317 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:12:51.540 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:25:15.412 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:12:51.678 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:25:15.417 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:12:51.640 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:29.361 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:18:05.627 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:29.374 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:18:05.597 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:29.376 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:18:05.600 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:29.379 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:18:05.603 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:29.380 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:18:05.604 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:29.388 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:18:05.655 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:29.574 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:18:05.798 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:29.589 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:18:05.812 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:29.656 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:18:05.922 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:31:40.643 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:19:16.909 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:31:40.684 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:19:16.950 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:31:40.700 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:19:16.924 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:31:40.724 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:19:16.948 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:31:40.725 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:19:16.949 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:31:40.741 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:19:16.965 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:31:40.760 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:19:17.026 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:31:40.784 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:19:17.008 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:31:40.792 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:19:17.016 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:35:39.663 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:23:15.928 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:35:39.674 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:23:15.940 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:35:39.683 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:23:15.907 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:35:39.689 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:23:15.913 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:35:39.689 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:23:15.913 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:35:39.699 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:23:15.920 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:35:39.882 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:23:16.106 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:35:39.895 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:23:16.161 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:35:39.911 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:23:16.135 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:03.157 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:27:39.423 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:03.168 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:27:39.434 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:03.245 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:27:39.468 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:03.248 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:27:39.472 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:03.257 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:27:39.481 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:03.259 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:27:39.483 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:03.379 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:27:39.602 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:03.400 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:27:39.666 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:03.409 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:27:39.632 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:51.640 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:28:27.907 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:51.648 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:28:27.914 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:51.662 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:28:27.886 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:51.669 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:28:27.892 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:51.679 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:28:27.903 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:51.700 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:28:27.924 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:51.715 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:28:27.980 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:51.743 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:28:27.966 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:40:51.791 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:28:28.014 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:44:34.650 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:32:10.916 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:44:34.664 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:32:10.931 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:44:34.736 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:32:10.960 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:44:34.739 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:32:10.962 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:44:34.780 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:32:11.004 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:44:34.783 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:32:11.007 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:44:34.917 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:32:11.184 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:44:34.956 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:32:11.177 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:44:34.980 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:32:11.204 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:46:02.497 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:33:38.763 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:46:02.508 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:33:38.774 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:46:02.515 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:33:38.739 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:46:02.517 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:33:38.741 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:46:02.522 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:33:38.747 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:46:02.535 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:33:38.759 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:46:02.610 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:33:38.834 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:46:02.636 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:33:38.860 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:46:02.642 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:33:38.909 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:25.802 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:36:02.069 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:25.880 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:36:02.146 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:25.934 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:36:02.158 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:25.957 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:36:02.181 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:25.959 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:36:02.183 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:25.960 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:36:02.184 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:25.989 INFO:tasks.ceph.mon.b.smithi022.stderr:==00:11:36:02.255 8485== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:26.088 INFO:tasks.ceph.mon.c.smithi035.stderr:==00:11:36:02.312 8862== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:26.091 INFO:tasks.ceph.mon.a.smithi035.stderr:==00:11:36:02.315 8861== Warning: unimplemented fcntl command: 1036 2019-12-09T20:50:50.240 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2019-12-09T20:50:50.241 DEBUG:teuthology.task.console_log:Killing console logger for smithi022 2019-12-09T20:50:50.241 DEBUG:teuthology.task.console_log:Killing console logger for smithi035 2019-12-09T20:50:50.241 DEBUG:teuthology.task.console_log:Killing console logger for smithi022 2019-12-09T20:50:50.242 DEBUG:teuthology.task.console_log:Killing console logger for smithi035 2019-12-09T20:50:50.242 DEBUG:teuthology.exit:Finished running handlers